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PREFACE 


This  Note  describes  a  study  funded  under  Contract  PRA-84-00689  with 
the  Division  of  Policy  Research  and  Analysis  (PRA)  of  the  National 
Science  Foundation  (NSF) .  The  purpose  of  the  project  is  to  identify  and 
assess  ways  to  improve  the  transfer  to  potential  users  of  knowledge 
generated  by  federally  funded  research  in  science  and  technology.  To 
accomplish  this  purpose,  the  study  examines  problems  of  information 
quality  control  and  processes  by  which  scientific  and  technical 
knowledge  is  or  could  be  tailored  and  packaged  for  users.  It  draws 
heavily  from  the  literature  listed  in  the  bibliography  and  from  informal 
telephone  conversations  with  federal  officials,  industrial  users, 
information  producers,  and  others.  The  study  includes  an  overview  and 
evaluation  of  current  federal  policies  and  practices,  and  an  assessment 
of  alternative  policy  options,  especially  as  they  may  relate  to  the  NSF. 
Appendixes  suggest  future  directions  for  research  in  dissemination 
policy  and  present  a  history  of  the  evolution  of  relevant  federal 
policies  and  milestones. 

In  view  of  the  two-month  duration  of  the  study,  this  report  does 
not  contain  a  formal  evaluation  of  current  policies  and  practices  or  a 
detailed  analysis  of  the  relative  merits  of  the  options  presented.  Nor 
does  it  formulate  a  comprehensive  plan  that  recommends  and  supports 
remedial  actions.  Rather,  its  intent  is  to  provide  a  framework  for 
longer  term  research  and  analysis  needed  in  this  field. 

Among  the  actions  feasible  today,  current  computer  network 
technology  could  be  used  to  link  the  nation's  scientific  community  and 
thereby  substantially  augment  the  information  transfer  process.  The 
technology- intensive  options  discussed  here  move  in  this  direction. 
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SUMMARY 


In  this  study,  we  consider  ways  to  more  effectively  transfer  to 
potential  users  the  knowledge  produced  by  federally  funded  research  in 
science  and  technology.  Federal  policymakers  have  been  concerned  that 
the  information  created  through  the  billions  of  dollars  spent  annually 
by  the  federal  government  is  not  well  utilized  because  of  inadequacies 
in  information  transfer  between  the  research  and  user  communities. 

Recent  literature  suggests  a  number  of  problems  contributing  to 
this  concern.  First,  the  very  low  level  of  support  for  knowledge 
transfer  in  comparison  to  knowledge  production  suggests  that 
dissemination  efforts  are  not  viewed  as  an  important  component  of  the 
R&D  process.  Second,  there  are  mounting  reports  from  users  about 
difficulties  in  getting  appropriate  information  in  forms  useful  for 
problem-solving  and  decision-making.  Third,  rapid  advances  in  many 
areas  of  science  and  technology  (e.g.,  biotechnology,  computer  science) 
can  be  fully  exploited  only  if  they  are  quickly  translated  into  further 
research  and  application.  Such  translation  requires  multidisciplinary, 
problem-focused  communication  of  scientific  information.  Traditional 
transfer  mechanisms  do  not  provide  that  kind  of  communication.  Finally, 
while  the  United  States  continues  to  be  seen  as  dominant  in  basic 
research  and  development,  concerns  have  arisen  that  foreign  competitors 
may  be  better  able  to  apply  the  results. 

The  federal  government  has  no  coherent,  centrally  organized,  or 
systematically  designed  approach  to  deal  with  disseminating  information 
created  by  the  basic  research  community.  Approaches  to  information 
transfer  vary  considerably  from  agency  to  agency,  and  within  any  given 
agency  they  have  changed  significantly  over  time.  These  variations  have 
reflected  changes  in  society's  attitudes  toward  science  and  technology, 
differences  between  agencies  in  how  they  view  their  mission,  and 
budgetary  opportunities  and  constraints. 

To  date,  federal  agencies  have  attempted  to  increase  the  flow  of 
information  chiefly  by  improving  its  availability.  Agencies  have 
provided  support  for  publication  of  results  in  refereed  scientific  and 
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technical  journals  and  have  sponsored  technical  reports  as  a  means  of 
quickly  disseminating  detailed  research  results.  More  recently, 
research-funding  agencies  and  federal  information  clearinghouses  have 
created  fully  indexed  and  abstracted  bibliographic  databases  with  online 
search  capabilities.  Many  government  information  services  have 
coordinated  their  efforts  with  the  private  sector,  particularly  with 
search  services  run  by  professional  societies. 

While  some  problems  remain  in  assuring  the  availability  of 
information,  federal  efforts  in  this  regard  have  been  largely 
successful.  As  a  result,  applied  scientists,  decisionmakers,  and  other 
information  users  are  now  faced  with  an  enormous  quantity  of  potentially 
pertinent  research  results  available  in  published  form  and  through 
online  services. 

However,  two  problems  remain.  First,  mechanisms  are  inadequate  to 
help  the  user  assess  the  quality  of  available  information.  Second,  the 
characteristics  of  actual  usage  behavior  are  not  sufficiently  taken  into 
account,  so  that  retrieval  of  relevant  information  in  forms  useful  for 
application  is  impaired.  Difficulties  include  the  lack  of  interactive 
knowledge  transfer  systems  and  "cultural"  differences  between 
information  producers  and  potential  users. 

Our  literature  review  and  informal  discussions  with  knowledgeable 
individuals  suggest  that  users  rely  on  two  major  systems  for  acquiring 
scientific  and  technical  information:  formal  document  distribution  and 
informal  contacts  with  colleagues.  Formal  search  services  often 
comprehensively  retrieve  voluminous  materials,  but  they  may  fail  to 
identify  those  of  greatest  significance.  The  interactive  and  informal 
collegial  system  is  more  timely  in  providing  information  about  current 
research  and  its  potential  significance  and  is  therefore  better  focused 
on  user  needs.  But  it  is  not  necessarily  comprehensive,  since 
individuals  are  unaware  of  information  resources  that  do  not  enter  their 
collegial  networks.  However,  formal  and  informal  systems  do  complement 
each  other.  For  example,  colleagues  are  often  used  as  pointers  to  the 
published  literature;  and,  for  those  entering  a  new  subject  field,  the 
literature  can  provide  names  to  contact  for  referral  to  more  recent 
work . 
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In  sum,  a  need  exists  for  more  interactive,  user-guided  information 
transfer.  Passive  access  to  voluminous  stores  of  textual  and  numeric 
data  does  little  to  promote  timely,  selective  retrieval  and  application 
of  high-quality  R&D  results.  Therefore,  we  propose  a  number  of  options 
for  improving  information  transfer  in  which  the  emphasis  is  on 
increasing  the  selectivity  and  user-responsiveness  of  existing  formal 
dissemination  systems  and  on  coupling  them  more  closely  to  informal 
systems . 


Technology-Intensive  Options-- These  alternatives  take  advantage 
of  the  special  properties  of  interactive  computer  and 
communications  technologies.  They  include: 

Experimental  programs  using  electronic  interactive 
networks  to  encourage  (a)  self -configured  groups  to 
pass  on  significant  new  research  findings  and  in  other 
ways  to  carry  on  functions  now  performed  by  informal 
contact  circles;  (b)  potential  users  to  remain  in 
contact  with  researchers  throughout  the  course  of  the 
project;  (c)  more  extended  information  research 
sharing,  such  as  exchange  of  primary  data  or  preprints; 
and  (d)  interconnection  of  networks  for  a  national 
decentralized  information  sharing  capability. 
Experimental  programs  to  support  the  design  and 
development  of  more  flexible  and  powerful  user-guided 
methods  for  retrieving  and  examining  highly  structured 
textual  and  numerical  data  bases. 

Supply  Side  Options-- These  would  seek  to  improve  dissemination 
processes  from  the  information  supply  side,  either  with  or 
without  electronic  technology.  They  include: 

Improvements  in  the  organization  of  information  by  (a) 
providing  abstracts  with  more  complete  research 
information,  (b)  organizing  information  by  clusters 
through  use  of  citation  indexes  and  current 
bibliometric  techniques;  (c)  organizing  abstracts  and 
key  words  "conceptually,"  with  terms  from  different 
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disciplines  serving  as  pointers  to  common  knowledge 
domains . 

Improvements  in  the  quality  of  data,  through  broader 
use  of  internationally  accepted  standards  of  evaluation 
and  other  mechanisms . 

Greater  use  of  information  intermediaries,  including 
encouraging  representatives  of  promising  research 
projects  to  provide  results  and  technical  assistance  to 
potential  users. 

•  User-Focused  Options-- These  are  directed  to  strengthening  the 
ability  of  users  themselves  to  access  and  evaluate  information. 
They  include: 

Building  user  skills  to  seek  out  and  find  relevant 
information  sources,  through  development  of  appropriate 
curricula  in  educational  institutions  and  special 
assistance  to  small  businesses  and  local  government 
agencies . 

Programs  to  encourage  the  direct  participation  of  users 
in  the  information  transfer  process,  by  relying  on  user 
feedback  and  other  methods  during  the  course  of 
research  projects. 

Collaborative  research  programs  involving  government, 
industry  and  universities  to  provide  potential  users 
with  opportunities  for  interaction  throughout  the  life 
of  a  research  effort  to  increase  the  utility  of  the 
results . 


We  suggest  four  areas  for  further  study.  First,  choosing  wisely 
among  these  options  will  require  a  systematic  evaluation  based  on 
technological,  economic,  and  institutional  considerations,  and  an 
analysis  of  the  roles  to  be  played  by  the  public  and  private  sectors. 
Second,  to  ensure  that  federal  information  transfer  policy  meets  the 
evolving  needs  of  users  will  require  further  research  to  assess  the 
potential  of  new,  especially  interactive,  technologies.  Third,  the 
regulatory  and  legal  environment,  including  the  effects  of  copyright  and 
antitrust  considerations,  merits  continuing  investigation.  Fourth,  the 
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effects  of  international  information  flow  and  the  experiences  of  other 
nations  in  implementing  information  transfer  policies  might  shed  light 
on  problems  and  opportunities  in  the  United  States. 

During  the  course  of  our  work,  we  have  become  convinced  that  two 
sorts  of  future  research  are  important.  First,  policy  research  in  the 
four  areas  we  have  outlined  is  needed  to  assess  the  context  in  which  any 
future  options  will  be  implemented.  Second,  exploratory  research  and 
development  is  needed  to  take  advantage  of  the  potential  inherent  in 
interactive  information  technology  for  the  improvement  of  scientific  and 
technical  knowledge  transfer. 
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I.  INTRODUCTION 


In  this  study,  we  consider  how  to  transfer  more  effectively  to 
potential  users  the  scientific  and  technical  knowledge  produced  by 
federally  funded  research  and  development.  Federal  policymakers  have 
been  concerned  that  the  information  created  through  the  billions  of 
dollars  spent  annually  by  the  federal  government  is  not  well  utilized 
because  the  transfer  process  between  the  research  and  user  community  is 
inadequate . 

Recent  literature  suggests  a  number  of  problems  contributing  to  the 
concern  over  shortcomings  in  the  information  transfer  process.  First, 
the  very  low  level  of  support  for  knowledge  transfer  in  comparison  to 
knowledge  production  suggests  that  dissemination  efforts  are  not  viewed 
as  an  important  component  of  the  research  and  development  (R&D) 
process.  Second,  there  are  mounting  reports  from  users  about 
difficulties  in  getting  appropriate  information  in  forms  useful  for 
problem-solving  and  decisionmaking.  Further,  rapid  advances  in  many 
areas  of  science  and  technology  (e.g.,  biotechnology,  computer  science) 
can  be  fully  exploited  only  if  they  are  quickly  translated  into  further 
research  and  application.  Such  translation  requires  multidisciplinary, 
problem-focused  communication  of  scientific  information.  Traditional 
sub j ect - focused  transfer  mechanisms  do  not  provide  that  kind  of 
communication.  Finally,  while  the  United  States  continues  to  be  seen  as 
dominant  in  basic  research  and  development,  concerns  have  arisen  that 
foreign  competitors  may  be  better  able  to  apply  the  results . 

To  date,  inadequacies  in  the  information  transfer  process  have 
fallen  into  three  categories.  The  first  is  in  information  availability- - 
placing  research  results  and  conclusions  in  media  intended  to  be 
accessible  to  the  user  community  at  large.  Because  information 
availability  is  of  such  obvious  importance,  it  has  been  discussed 
extensively  in  the  literature  and  promoted  through  federal  policy.  In 
response,  many  databases  of  varying  utility  have  been  created  for 
storage,  search,  and  retrieval  of  scientific  and  technical  information. 
Consequently,  inadequacies  in  information  availability  have  been  largely 
rectified. 
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The  second  category  of  shortcomings  are  those  in  quality  assurance-- 
enhancing  the  ability  of  users  to  evaluate  the  scientific  merit  of 
information  they  receive.  Here  concerns  focus  on  the  accuracy  of 
primary  data  and  the  validity  and  scope  of  conclusions  drawn  from  them. 
The  rudimentary  and  restricted  nature  of  current  quality  assurance 
efforts  compromises  the  abilities  of  scientists  and  engineers  to 
retrieve  sound  information  from  the  vast  quantities  of  available  data 
and  to  build  on  what  others  have  learned. 

The  third  category  is  user-responsiveness- -organizing  and 
communicating  information  in  ways  that  take  into  account  the 
characteristics  of  potential  users  and  make  available  information  more 
useful  and  easier  to  find.  Problems  arise  in  part  because  of  the  lack 
of  an  interdisciplinary  information  structure  and  of  a  language  common 
to  information  producers  and  users,  and  in  part  because  of  the  lack  of 
an  interactive  knowledge  transfer  system. 

Because  so  much  attention  has  been  devoted  to  information 
availability,  chiefly  concerned  with  questions  of  access,  we  will 
concentrate  on  the  second  and  third  categories,  which  are  critically 
important  and  yet  have  received  rather  little  attention. 

METHODOLOGY 

This  study  draws  primarily  from  two  sources: 

•  An  extensive  survey  of  the  literature,  including  items  listed 
in  the  bibliography. 

•  Telephone  or  face-to-face  discussions  with  knowledgeable 
individuals,  including  federal  information  managers,  university 
professors  or  senior  scientists  in  research  institutions, 
executive-level  representatives  of  information-using 
industries,  representatives  of  firms  in  the  information  supply 
industry,  senior  staff  in  government  funding  agencies,  and 
librarians . 
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DEFINITIONS  OF  KEY  TERMS 

"Scientific  and  technical"  information  includes  new  knowledge 
generated  by  R&D  activities;  it  may  appear  as  text,  numeric  data, 
models,  proofs,  prototypes,  and  the  like.  We  are  concerned  in  this 
study  with  nonproprietary  information  that  originates  from  R&D. 

"information  producers"  or  "suppliers"  are  those  who  carry  out  R&D 
activities.  In  the  main,  these  activities  have  been  conducted  by 
universities,  research  institutes,  and  R&D  divisions  of  major 
corporations . 

"information  users"  or  "consumers"  constitute  a  heterogeneous  and 
potentially  enormous  population.  For  purposes  of  this  study  the  user 
population  is  divided  into  four  categories: 

•  The  research  community  itself,  as  it  builds  new  efforts  on  the 
results  of  previous  research  and  development. 

•  Government  at  all  levels,  as  it  relies  on  scientific  and 
technical  information  to  support  decisionmaking. 

•  Industry,  as  it  applies  scientific  and  technical  knowledge  to 
the  development  of  new  products  and  processes  and  to  the 
improvement  of  extant  ones . 

•  The  general  public,  as  it  brings  disseminated  information  to 
bear  on  a  range  of  professional,  social  and  personal  issues. 

In  this  study,  we  consider  information  transfer  to  the  first  three 
groups.  The  interests  and  requirements  of  the  general  public  are  much 
more  varied  than  those  of  the  other  groups  and  would  merit  a  separate 
study  to  be  treated  adequately. 

"Information  media"  are  the  material  or  technical  means  by  which 
R&D  results  are  expressed.  Traditionally,  print  media  have  been  used  to 
convey  "primary"  information  products  such  as  research  reports,  journal 
articles,  and  monographs,  as  well  as  "secondary"  products  such  as  review 
articles,  handbooks,  and  bibliographies.  In  recent  years,  electronic 
storage  has  been  increasingly  employed,  particularly  for  secondary 
products  such  as  bibliographies,  abstracts  and  indexes.  Finally,  people 
themselves  should  be  regarded  as  media;  a  great  deal  of  scientific  and 
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technical  information  is  communicated  in  conferences  and  classrooms,  and 
via  collegial  contacts  and  manpower  flow. 

"Transfer"  or  "dissemination"  refers  to  efforts  to  move  the 
products  of  R&D  from  sources  to  users.  The  objective  is  knowledge 
utilization:  the  translation  of  scientific  and  technical  information 

into  new  research  and  development,  industrial  process  or  product 
applications,  and  informed  policy  decisions.  Concerns  about  the 
adequacy  of  information  transfer  or  dissemination  encompass  the  three 
components  noted  above- -availability ,  quality  assurance,  and  user 
responsiveness . 

ORGANIZATION  OF  THIS  NOTE 

Section  II  provides  an  overview  and  evaluation  of  current  federal 
policies.  In  Section  III  we  discuss  options  for  improving  the  transfer 
of  scientific  and  technical  information,  along  with  the  merits  of  those 
options.  Appendix  A  contains  suggestions  for  future  research  in 
dissemination  policy,  including  a  set  of  illustrative  criteria  for 
formally  evaluating  current  programs  and  future  options.  Appendix  B 
describes  the  history  of  past  federal  policies  relating  to  information 
availability,  quality  assurance,  and  responsiveness  to  the  user 
environment.  Appendix  C  lists  important  milestones  in  the  history  of 
information  poilcy. 
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II.  OVERVIEW  AND  EVALUATION 
OF  CURRENT  FEDERAL  POLICIES 


The  literature  review  and  discussions  conducted  as  part  of  this 
study  suggest  that  information  users  rely  on  two  major  systems  for 
acquiring  scientific  and  technical  information:  collegial  contact 
networks  and  formal  document  distribution  (cf.  Cronin,  1982;  Gerstenfeld 
and  Berger,  1980).  Peer  interactions  have  assumed  increasing  importance 
for  information  transfer  as  research  documents  and  retrieval  mechanisms 
have  proliferated.  Formal  search  services  often  retrieve  a  lot  of 
material  but  fail  to  indicate  what  is  of  greatest  significance. 

However,  formal  and  informal  systems  do  complement  each  other.  For 
example,  colleagues  are  often  used  as  pointers  to  the  published 
literature,  and  for  those  entering  a  new  subject  field,  the  literature 
can  provide  names  to  contact  for  referral  to  more  recent  work.  Formal 
documents  yield  a  comprehensive  historical  archive  of  findings,  while 
collegial  interactions  are  better  sources  of  information  about  current 
research  and  its  potential  application. 

This  section  summarizes  the  status  and  achievements  of  some  of  the 
more  formal  information  transfer  systems  that  federal  policies  encourage 
or  rely  on.  Many  problems  remain  with  formal  systems,  and  the 
difficulties  informal  systems  face  in  filling  the  gap  are  also  assessed. 
Information  policies  and  practices  are  evaluated  in  terms  of  the  extent 
to  which  they  assure  availability,  enable  quality  assessment,  and 
respond  actively  to  the  needs  of  users. 

AVAILABILITY  ASSURANCE 
Status  and  Achievements 

The  two  primary  formats  for  release  of  federal  research  results  are 
the  scientific  or  technical  journal  and  the  technical  report.  The  form 
chosen  depends  on  the  nature  of  the  funding  agency  and  the  tradition  of 
the  discipline  in  which  research  is  funded.  NSF,  for  example,  serves 
the  scientific  community  primarily  through  support  of  basic  research. 

Its  awards  often  go  to  investigators  in  academic  research  organizations. 
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Similarly,  NIH  operates  within  the  long-established  traditions  of  the 
biomedical  research  discipline.  Both  agencies  prefer  the  scientific 
journal  as  the  instrument  for  reaching  their  audiences,  although  they 
also  produce  a  substantial  number  of  technical  reports. 

The  major  mission-oriented  agencies  like  the  Department  of  Energy, 
the  National  Aeronautics  and  Space  Administration,  and  the  Department  of 
Defense  contract  with  large  private  sector  organizations  as  well  as  with 
universities  and  small  businesses.  They  focus  on  more  specific  problems 
with  heavier  technical  emphasis.  Traditionally,  the  government  report 
serves  them  as  both  the  releasing  mechanism  and  a  contractual  record. 
Technical  reports  play  an  important  role  in  providing  the  details  of 
procedures  and  results.  Under  the  pressure  of  a  steadily  increasing 
research  volume  and  rising  publication  costs,  journal  publishers  usually 
do  not  publish  detailed  information  on  experimental  design,  test 
instrumentation,  or  experimental  findings  necessary  for  data  evaluation. 
Technical  reports  have  no  space  limits  and  often  serve  as  a  permanent 
record  of  project  activities  and  results. 

Most  of  the  large  federal  information  services,  such  as  the 
National  Technical  Information  Service  or  the  Department  of  Energy's 
Technical  Information  Center,  have  developed  in  part  to  archive  and 
disseminate  technical  report  literature.  These  services  are  thus 
analogous  to  those  provided  by  traditional  scientific  bibliographic  and 
document  delivery  channels  for  the  scientific  journal  and  other  refereed 
sources.  Through  these  functions,  mission  agencies  have  pioneered  the 
development  of  computerized  information  handling  and  retrieval  systems. 
Many  of  these  systems  include  large,  abstracted  files  of  the  refereed 
journal  and  book  literature  as  well.  The  justification  for  these 
government  services  in  areas  already  served  by  the  private  sector-- 
besides  the  fact  that  so  much  federal  research  is  published  in  journals 
and  books--has  been  the  need  to  collect  mission-oriented  literature  in 
one  place. 

Current  transfer  policies  and  practices  are  largely  successful  at 
making  scientific  and  technical  information  available.  Research 
findings  are  published,  indexed,  and  abstracted.  Publishers,  government 
central  depositories,  and  libraries  supply  documents  on  request. 
Abstracts  and  indexes  have  proliferated.  Many  of  them  have  been  put 
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online  and  are  available  worldwide.  Librarians  and  specialists  in  using 
online  search  services  take  advantage  of  these  systems  to  locate 
materials  requested  by  users . 

It  is,  of  course,  never  certain  that  all  research  is  reported.  For 
example,  several  persons  who  took  part  in  our  discussions  mentioned 
difficulties  locating  government  reports  to  which  they  had  found 
references.  However,  in  view  of  the  strong  incentives  for  scientists  to 
gain  recognition  for  their  work  (Glaser,  1980b),  the  general  belief  is 
that  all  significant  results  get  into  publication.  The  consensus  from 
sources  we  reviewed  is  that  the  United  States  is  exceedingly  rich  in 
scientific  and  technical  literature  and  that  bibliographic  search 
services,  although  not  fully  perfected,  have  proved  their  utility  in 
"making  a  first  pass  through  the  information  morass"  (Lide,  1981;  cf. 
Surprenant,  1982).  Appendix  B  presents  a  more  thorough  description  of 
the  evolution  of  federal  policies  and  practices  in  the  area  of 
information  availability. 

Problems 

Nonetheless,  a  number  of  problems  remain  in  assuring  the 
availability  of  scientific  and  technical  information.  We  have 
identified  five  principal  ones: 

•  Many  potential  users  are  unaware  of  much  of  the  information 
available  to  them. 

•  Many  potential  users  are  discouraged  by  the  complexity  and  cost 
of  obtaining  information  from  abstracting  and  indexing 
services . 

•  Dissemination  of  research  products  is  narrow. 

•  Formal  information  transfer  systems  are  not  timely  enough. 

•  The  continuity  of  abstracting  and  indexing  systems  is 
threatened  by  budgetary  constraints. 

Lack  of  Awareness.  A  General  Accounting  Office  survey  of 
managers  in  government  information  centers  revealed  that  63  percent  knew 
about  scientific  and  technical  databases  duplicating  or  overlapping 
their  own.  The  report  attributes  this  condition  in  part  to  lack  of 


awareness  of  existing  databases  when  creating  new  ones  (GAO,  1979; 

David,  1980).  Further,  Robbin  (1981),  Sprehe  (1981),  and  others  have 
found  that  the  federal  statistical  system,  which  contains  census  and 
health  data,  is  seriously  underutilized  both  inside  and  outside  the 
federal  government,  primarily  because  of  the  lack  of  information  about 
available  databases.  Agency  personnel  who  work  with  data  users  report 
that  "for  every  actual  data  user  they  encounter  they  find  many  others 
who  need  Federal  statistics  but  are  unaware  that  they  are  available  from 
the  Federal  Government"  (Robbin,  1981)  or  else  are  unable  to  find  out 
where  the  information  exists  (GAO,  1979).  Finally,  it  is  likely  that 
small  firms  and  local  government  agencies  are  at  a  disadvantage  in 
obtaining  scientific  and  technical  information;  moreover,  research 
suggests  they  are  in  the  main  unaware  of  the  serious  information  gap 
they  face  (Streetman,  1979;  Rothwell,  1979). 

System  Complexity  and  Cost.  Several  sources  attest  to  the 
burgeoning  complexity  and  cost  of  the  abstracting  and  indexing  systems 
created  to  make  information  retrieval  simpler  and  easier. 

•  Addressing  the  American  Chemical  Society  in  1976,  Graham 
pointed  out  that  a  Chemical  Abstracts  subscription  cost  about 
$3500  and  included  over  450,000  items  per  year.  In  1950,  the 
cost  had  been  about  $300  a  year  for  50,000  abstracts.  By  now, 
a  Chemical  Abstracts  subscription  costs  $62,000  annually  and 
includes  34,000  new  items  each  month.  Such  changes  have 
required  users  to  dramatically  alter  their  strategies  for 
becoming  or  staying  well  informed  in  a  field. 

•  In  1975,  the  Human  Interaction  Research  Institute  published  a 
guide  to  information  sources  in  the  mental  health  field.  It 
described  26  different  search  services,  20  separate  indexes  to 
the  periodical  literature,  and  25  reviews  (digests, 
newsletters,  journals)  for  keeping  current.  A  foreword  by  the 
chief  of  mental-health  services  development  at  NIMH  commented, 
"Most  of  us  on  the  policy  and  practice  side  are  at  least 
vaguely  aware  that  there  are  ways  to  reach  into  the  stockpile 
of  information.  We  mumble  the  acronyms  and  alphabet  letters  of 
retrieval  resources.  But  how  remote  and  unavailable  they  often 


remain. 
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•  Speaking  for  government  administrators,  McGowan  and  Loveless 

(1981)  state,  "Public  agencies  now  find  themselves ...  trying  to 
hold  the  line  on  service  costs  while  incurring  additional 
expenses  for  gathering  and  utilizing  information."  Concurring, 
Bozeman  and  Cole  (1982)  note  that  scientific  and  technical 
information  is  voluminous,  fragmented,  conflicting,  and 
obscure.  The  costs  of  access  for  public  managers,  in  terms  of 
time  and  effort  as  well  as  budget,  "are  such  that  a  common 
'solution'  is  simply  to  ignore  relevant  [scientific  and 
technical]  information." 

Consistency,  clarity,  and  appropriateness  of  key  words  and  formats 
for  searching  bibliographic  databases  are  mentioned  as  serious  problems 
(e.g.,  Wilkin,  1974;  Robbin,  1981).  For  example,  in  a  large-scale 
survey  of  engineers  and  scientists  employed  at  five  major  U.S. 
corporations,  nearly  half  the  respondents  mentioned  difficulties  with 
search  formats,  retrieval  structures,  ability  to  use  key  words  to  get 
desired  information,  and  other  systems  access  issues  (Gerstenfeld  and 
Berger,  1980).  As  Lide  notes,  with  bibliographic  systems,  "a  user  must 
learn  the  language  and  protocols  peculiar  to  each  service"  (1981). 

Other  problems  arise  in  relation  to  abstracts.  Sometimes  they  are  not 
available  at  all,  and  in  other  instances  they  are  so  incomplete  as  to  be 
of  little  more  value  than  the  title  itself  in  indicating  what  kind  of 
information  is  available  in  the  indexed  material  (Thomas,  1982).  More 
careful  attention  to  abstracting  by  substantively  trained  individuals 
was  recommended  in  several  of  our  telephone  discussions. 

Obtaining  and  using  numeric  databases  pose  "a  monumental  challenge 
to  the  scientist  or  engineer  (Hawkins,  1980).  For  machine-readable 
numeric  databases,  important  obstacles  to  availability  include  complex 
file  structures,  varied  formats,  and  the  need  to  write  the  data  to 
specifications  for  a  particular  user  environment  (Lide,  1981);  such 
difficulties  are  detailed  in  user  studies  conducted  by  Robbin  (1981)  and 
David  (1980).  Bibliographic  search  services  can  provide  only  limited 
assistance  in  locating  uncompiled  data,  because  the  existence  of  the 
data  typically  is  not  mentioned  in  the  abstract  of  a  paper  and  authors 


10 


do  not  know  how  to  cite  machine-readable  files  (Robbin,  1981;  Hawkins, 
1980).  Consequently,  uncompiled  data  sources  are  often  learned  of  only 
through  informal  collegial  contacts. 

To  take  advantage  of  online  information  systems,  the  user  also  has 
to  be  assured  of  the  following: 

•  Availability  of  a  computer  terminal. 

•  Adequate  connect  time. 

•  Subscriptions  to  an  array  of  bibliographic  services. 

•  Skill  in  using  the  services  (either  directly  or  via  an 
information  professional). 

•  Ability  to  acquire  an  item  of  information  once  it  has  been 
identified . 

•  Funds  to  cover  the  expenses  that  these  efforts  entail  (in 
labor,  equipment,  and  services).  Declining  hardware  costs 
ameliorate  some  of  these  problems,  but  start-up  expenditures 
and  labor  intensive  services  remain  serious  issues. 

All  of  these  requirements,  together  with  system  shortcomings, 
constitute  a  formidable  set  of  access  limitations  for  many  potential 
users  of  the  body  of  scientific  and  technical  information  that  is,  in 
principle,  publicly  available. 

Narrowness  of  Dissemination.  Research  results  tend  to  be 
disseminated  within  small,  homogeneous  audiences  (Bozeman  and  Cole, 

1982)  or,  as  one  knowledge  transfer  expert  told  us,  within  scientific 
and  technical  subdisciplines.  The  reasons  for  this  are  fairly  clear. 
First,  formal  information  transfer  channels --including  both  primary 
product  distribution  and  secondary  information  resources --tend  to  be 
organized  along  traditional  disciplinary  lines  (Tornatzky  et  al.,  1983; 
Bikson,  1980).  Between-group  differences  in  language,  norms,  and  values 
create  communication  barriers  often  analogized  to  cultural  differences 
(e.g..  Rich,  1981).  Second,  collegial  contact  networks --which  in 
practice  seem  to  constitute  the  single  most  important  source  of 
referrals  to  extant  information- -are  also  organized  along  disciplinary 
lines  (e.g.,  Cronin,  1982).  Together,  the  two  systems  make  for 
effective  information  dissemination  within  fields  of  inquiry.  However, 
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dissemination  across  disciplines  does  not  occur  readily,  and  transfer  of 
information  from  academic  sources  to  nonacademic  audiences  is  even  more 
impaired  (Tornatzky  et  al.,  1983). 

Sometimes  dissemination  systems  work  within  boundaries  even  more 
restricted  than  those  of  a  subdiscipline.  For  example,  technical 
reports  produced  by  private  laboratories  usually  receive  no  circulation 
outside  the  sponsoring  organization.  Reports  produced  within  academic 
or  public  research  institutes  may  receive  wide  initial  distribution  but 
rarely  enter  formal  channels  for  archiving  and  future  delivery 
(Rossmassler  and  Watson,  1980). 

Further,  attempts  to  broaden  dissemination  often  fail  if  user  needs 
are  not  taken  into  account.  One-way,  untargeted  distribution  efforts 
may  only  result  in  the  accumulation  of  unused  materials  at  the  receiving 
end  (cf.  Rich,  1981;  Bikson,  1980).  Many  of  the  persons  with  whom  we 
spoke  said  this  was  a  common  fate  for  unsolicited  reports,  which  seemed 
to  arrive  "in  truckloads"  and  were  quickly  shelved.  We  found  no  studies 
that  reported  the  proportion  of  potential  users  of  documents  who  in  fact 
used  them--clearly  a  difficult  estimation  task  (cf.  Robbin,  1981). 

Tardiness.  A  number  of  studies  have  demonstrated  that  information 
potentially  available  in  the  early  stages  of  projects  is  often  not  found 
until  too  late  to  be  useful  (cf.  Thomas,  1982).  However,  what  is  timely 
has  been  found  to  vary  depending  on  the  nature  of  the  information  need. 
It  is  generally  agreed  that,  for  users  who  are  themselves  engaged  in 
research  and  development,  the  results  of  others'  work  often  help  them 
improve  their  own  conceptualizations  and  can  be  of  value  even  very  late 
in  a  project.  On  the  other  hand,  for  users  who  are  engaged  in 
application  or  administration,  information  must  be  acquired  very  early 
if  it  is  to  be  helpful  (cf.  Bozeman  and  Cole,  1982;  Gerstenfeld  and 
Berger,  1980).  While  computer-based  methods  have  helped  to  expedite 
publication  and  updating  of  information  as  well  as  its  location  and 
retrieval,  the  process  typically  does  not  move  rapidly  enough  to  meet 
the  needs  of  applied  scientists  or  decisionmakers.  Reporting, 
publishing,  abstracting,  and  indexing  take  a  considerable  amount  of  time 
(Adam,  1975).  Thus,  informal  contact  networks  are  invoked  when 
information  needs  are  immediate.  By  these  channels,  users  receive 
preprints  of  reports  and  learn  the  most  recent  results  of  relevant  work 
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in  progress.  Timeliness,  then,  is  a  criterion  more  often  met  by 
informal  than  by  formal  information  transfer  procedures. 

Threats  to  Continuity.  Traditional  scientific  journals  provide 
continuous  access  to  research  knowledge.  Continuity  of  access  via 
secondary  information  sources  and  services  is  subject  to  several 
constraints.  The  first  is  the  fact  that  information  predating  the 
initial  coverage  dates  of  present  bibliographic  databases  can  be 
difficult  to  find.  Second,  many  sources  have  called  attention  to  the 
uncertainty  and  changeability  of  the  information  policy  environment  of 
which  dissemination  processes  are  a  part  (e.g.,  Nason,  1979;  Gibbons  and 
Guemmett,  1979;  National  Academy  of  Engineers,  1981).  Policy 
variability  creates  planning  problems  both  for  information  service 
providers  and  for  potential  users. 

Most  important,  however,  are  curtailments  of  services,  such  as  the 
termination  of  the  former  Smithsonian  Scientific  Information  Exchange 
and  the  earlier  Office  of  Scientific  Information  Services.  It  has 
recently  been  argued  that  federal  information  agencies  provide  tax- 
supported,  underpriced  services  that  could  be  more  efficiently  and 
fairly  supplied  by  the  private  sector,  with  the  users  paying  full  costs. 
Some  federal  databases  do  compete  with  private  sector  products,  and 
commercial  search  services  have  sometimes  bought  federal  databases  only 
to  find  it  difficult  to  sell  what  the  federal  services  price  so  low. 

The  distribution  of  responsibilities  between  the  private  sector  (for- 
profit  and  not-for-profit)  and  the  public  sector,  as  well  as  marketing 
philosophies  for  federal  information,  remain  unresolved. 

In  moving  toward  a  resolution,  it  is  important  to  recognize  that 
the  private  sector  plays  key  roles,  especially  in  providing  quality 
assurance  and  information  tailored  to  user  needs.  In  evaluating  current 
programs  and  future  options,  as  discussed  in  Appendix  A,  careful 
attention  should  be  paid  to  roles  the  federal  government  might  play  in 
strengthening  or  supplementing  activities  in  the  private  sector.  If  an 
orderly  transfer  of  responsibilities  is  not  arranged,  discontinuation  of 
federal  services  could  result  in  important  loss  of  access  to  research 
information. 
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QUALITY  ASSURANCE 
Status  and  Achievements 

The  historical  approach  to  quality  control  of  primary  literature 
has  been  peer  review.  Federal  research  results  published  in  scientific 
journals  enjoy  the  benefits  of  their  peer  review  process,  and  the 
professional  societies  that  sponsor  the  journals  are  regarded  as  doing  a 
creditable  job  of  substantive  evaluation  (e.g.,  Abelson,  1980;  Bozeman 
and  Kim,  1981).  Besides  the  reputation  of  the  journal  itself,  users 
also  treat  the  reputation  of  laboratories  or  institutions  responsible 
for  primary  research  products  as  indicators  of  their  quality.  Further, 
if  they  are  available,  scientific  review  articles  are  regarded  as 
important  evaluative  syntheses  of  primary  knowledge  within  a 
subdiscipline  (Glaser,  1980;  Huisman  and  Koster,  1978/79;  Branscomb, 
1968).  Because  such  efforts  are  costly  and  time-consuming  but  receive 
less  recognition  than  primary  scholarship,  they  are  not  widespread 
(several  discussants  recommended  greater  federal  agency  support  for 
critical  reviews).  Results  published  in  technical  reports  do  not 
receive  final  peer  review,  although  the  research  they  report  may  be 
reviewed  in  progress  (Passman,  1969). 

Outside  of  peer  review,  efforts  at  quality  control  or  evaluation 
are  limited.  Some  information  users  take  advantage  of  citation  indexes 
from  the  Institute  for  Scientific  Information  to  measure  the  impact  of 
published  work.  In  recent  years,  the  Institute  has  established  new 
databases  tracing  frontier  research  in  biomedicine,  geosciences,  and 
mathematics  through  groups  of  key  research  pieces.  Newer  literature  is 
measured  and  weighted  by  the  number  of  key  references  in  an  area  cited. 

Online  systems  have  made  some  quality  assurance  problems  with 
numeric  data  more  tractable;  for  example,  storage  and  updating  is  vastly 
more  efficient,  and  computerized  cleaning  procedures  can  be  employed. 
Coupling  these  services  with  online  retrieval  and  manipulation  by  users 
could  expedite  the  dissemination  of  high  quality  scientific  data  (David, 
1980;  Hawkins,  1980;  Golashvili,  1973);  however,  such  capabilities  have 
yet  to  be  designed  and  implemented.  In  the  meantime,  numeric 
information  systems  that  evaluate  and  repackage  research  data  facilitate 
the  use  of  data  beyond  the  purposes  for  which  it  was  originally 
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collected.  Although  numeric  databases  constitute  only  a  small  fraction 
of  scientific  and  technical  information,  their  potential  utility  is 
extremely  high  both  for  the  direct  answering  of  factual  questions  and 
for  the  testing  of  hypotheses  through  secondary  analysis.  But  for 
extant  data  to  serve  these  purposes,  strong  quality  assurance  is  needed. 

The  CODATA  classification  scheme  yields  three  broad  categories  of 
numeric  data  that  provide  a  foundation  for  quality  assessment:  measured 
properties  of  well-defined  systems  (e.g.,  most  chemical  properties 
data);  observational  or  environmental  data  (e.g.,  seismographic  data); 
and  statistical  data  (e.g.,  epidemiological  records).  There  are  clear 
standards  for  the  production  of  reliable  measures  within  well-defined 
systems  of  properties,  and  accepted  approaches  to  quality  control  of 
observational/environmental  data  through  control  and  documentation  of 
measuring  instruments  and  procedures.  Quality  control  of  statistical 
data  is  more  difficult,  and  there  is  not  widespread  agreement  as  to  what 
it  entails  (Lide,  1981).  Consequently,  quality  assurance  through  use  of 
evaluated  databases  and  information  analysis  centers  is  limited  to  some 
subsets  of  materials  properties  data. 

Data  evaluation  projects  require  coordinated  input  from  many 
sources.  Most  projects  to  improve  the  quality  of  scientific  and 
technical  data  involve  a  combination  of  players  --  government,  industry, 
professional  societies,  universities,  and  often  international 
institutions.  In  the  United  States,  the  leadership  role  in  government 
efforts  is  played  by  the  National  Academy  of  Science's  Numerical  Data 
Advisory  Board,  representing  the  U.S.  in  CODATA.  The  National  Bureau  of 
Standards'  Office  of  Standard  Reference  Data  supports  a  network  of  25 
data  evaluation  centers.  Professional  societies  such  as  the  American 
Institute  of  Chemical  Engineers  and  trade  associations  such  as  the 
American  Petroleum  Institute  also  support  some  data  evaluation  centers. 
Federal  funding  for  information  analysis  centers  (IACs),  where  some  data 
evaluation  is  carried  out,  has  waned  since  the  1960s  when  most  of  them 
were  established.  Industry  associations  still  support  some  IACs.  Some 
firms  have  established  data  collection  and  evaluation  centers,  for 
example,  General  Electric's  Electronic  Materials  Properties  Information 
Center  (EMPIS) . 
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The  opportunities  offered  by  computer  technology  make  expanded  data 
evaluation  efforts  more  feasible  and  more  attractive.  New  developments 
in  computer-aided  design  and  manufacture  require  reliable  data  for 
incorporation  into  design  analysis  programs.  The  Metals  Properties 
Council  is  supporting  the  development  of  a  National  Cooperative 
Materials  Property  Data  Network.  Still,  these  subfields  comprise  only  a 
minute  sector  of  the  area  of  materials  properties,  and  the  latter 
account  for  only  a  small  fraction  of  the  data  produced  by  scientific 
research.  Appendix  B  discusses  the  historical  background  of  federal 
policies  and  practices  directed  toward  assuring  the  quality  of  R&D 
information  products. 

Problems 

According  to  the  literature  we  reviewed  and  the  discussions  in 
which  we  participated,  information  users  regard  themselves  as  the  ideal 
evaluators  of  primary  research  products  in  their  fields  of  expertise. 
This  ideal,  however,  is  often  infeasible.  Many  of  the  most  challenging 
and  promising  opportunities  for  the  application  of  research  arise  from 
work  performed  outside  the  potential  user's  discipline  (McGowan  and 
Loveless,  1981).  For  primary  products  outside  users'  areas  of 
competence,  it  is  difficult  to  distinguish  good  research  from  literature 
that  sounds  "science- like"  (Bozeman  and  Bozeman,  1981);  the  latter,  in 
fact,  may  well  be  more  readable  (Beyer  and  Trice,  1982).  But  the  most 
important  obstacle  to  user  evaluation  is  the  sheer  quantity  of 
information  produced.  While  current  dissemination  policies  and 
practices  increase  the  amount  of  scientific  and  technical  information 
available  to  potential  users,  they  complicate  the  evaluation  of  its 
quality . 


According  to  Lide  (1981),  conflicts  and  inconsistencies  in 
available  information  often  delay  decisions  that  have  a  strong 
technical  component  or  else  result  in  decisions  being  made  on  a 
grossly  inadequate  basis.  "At  times  the  sheer  mass  of  data 
tends  to  discourage  a  logical  approach  to  decisionmaking." 
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•  Echoing  this  view  from  the  public-administration  perspective, 
Bozeman  and  Cole  (1982)  point  out  that  "the  hue  and  cry  about 
information  overload"  has  not  prevented  policymakers  from 
contending  that  they  lack  sufficient  good  information  on  which 
to  base  decisions. 

•  Many  discussants  noted  that  for  any  given  subject  field  the 
ratio  of  useful  to  useless  information  was  likely  to  be  very 
small.  In  a  similar  vein,  Surprenant  (1982)  comments  that 
American  society  "has  moved  from  being  information  poor  through 
information  rich,...  arriving  at  an  age  of  information  junk" 

(cf.  Branscomb,  1979). 

Bibliographic  databases.  For  the  most  part,  bibliographic  databases 
leave  the  user  without  effective  means  of  filtering  the  literature. 
Consequently,  users  with  whom  we  spoke  report  that  they  take  advantage 
of  general  literature  searches  rarely,  if  at  all.  Collections  of 
government  reports  and  collections  of  items  supplied  by  private  for- 
profit  services  are  regarded  as  especially  problematic  from  a  quality 
standpoint,  because  the  entries  have  not  been  subjected  to  a  recognized 
review  process  (cf.  Corwin  and  Louis,  1982).  A  university  researcher 
described  one  such  literature  search  that  yielded  1000  articles,  only 
five  of  which  turned  out  to  be  usable.  As  a  representative  from  a  large 
manufacturing  corporation  put  it,  "if  you  want  goldfish,  you  fish  in  a 
goldfish  pond;  you  don't  fish  in  the  ocean--you  get  too  much  of  what  you 

i  .  it 

can  t  use. 

Titles,  abstracts,  subject  indexes,  and  key  words  are  not  precise 
enough  to  permit  a  selective  look  at  the  literature,  especially  in  the 
social  sciences.  Most  discussants  thought  that  providing  abstracts  for 
all  research  with  more  complete  documentation  (especially  of 
methodology)  would  help  address  the  quality  problem.  In  addition,  they 
believed  that  improved  indexing  should  be  possible.  Currently, 
according  to  its  editor,  IEEE  Transactions  is  experimenting  with  both 
sorts  of  improvements. 
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Other  individuals.  If  they  have  the  manpower  and  budget  resources, 
users  prefer  to  have  other  individuals  serve  as  quality  filters.  Most 
university-based  users  and  several  industrial  representatives  commented 
that  graduate  students  work  very  well  for  this  purpose,  because  they 
have  substantive  training,  benefit  from  the  literature  search  as  a 
learning  experience,  and  are  relatively  inexpensive.  Policymakers  and 
some  industrial  users  cited  the  need  for  an  experienced  professional 
staff  to  sort  and  distill  relevant  literature;  however,  costs  often  make 
such  staffing  prohibitive  (cf.  McGowan  and  Loveless,  1981). 

Well-trained  reference  librarians  were  seen  as  vital  sources  of 
assistance  in  using  bibliographic  services  to  locate  primary  products, 
but  were  not  viewed  as  helpful  for  making  evaluative  discriminations 
among  them. 

Numeric  databases.  Formally  validated  data  programs  such  as  those 
for  materials  properties  data  or  the  National  Library  of  Medicine's 
Toxicology  Information  Program  are  expensive  and  concentrated  in  areas 
where  the  need  for  precise  numerical  values  predominate.  In  other 
areas,  the  quality  of  available  numeric  data  is  generally  agreed  to  be 
dubious  (Hawkins,  1980;  Adam,  1975).  Further,  much  improved 
documentation  is  needed  to  make  machine-readable  data  more  generally 
useful,  along  with  systematic  notation  about  variable  definitions, 
missing  values,  methodological  problems,  and  so  on  (see  Robbin,  1981  and 
David,  1980) . 

Sources  we  reviewed  essentially  concurred  with  the  conclusion  of  a 
1978  National  Academy  of  Sciences  finding  that  U.S.  spending  on  data 
evaluation  is  disproportionately  small  in  comparison  with  amounts  spent 
on  generating  it  (cf.  Branscomb,  1983).  Nationwide,  support  for 
evaluation  and  dissemination  of  numeric  data  has  been  estimated  at 
between  six  and  ten  million  dollars  a  year  (NAS/NRC  Committee  on  Data 
Needs,  Roth  S/T  Policy).  In  contrast,  it  is  estimated  that,  in  the 
chemical  industry  alone,  between  $6  billion  and  $10  billion  could  be 
saved  each  year  in  capital  costs  and  operating  costs  if  data  needs  were 
adequately  met. 
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Although  feasible,  implementation  of  technology-intensive  solutions 
to  data  quality  problems  face  serious  difficulties  because  of  high  start¬ 
up  costs  and  operating  fees  as  well  as  intersystem  incompatibilities. 

Also,  where  implementation  costs  have  been  borne  by  the  private  sector, 
access  has  often  been  restricted.  For  example,  even  though  the  bulk  of 
primary  material  collected  by  General  Electric's  EMPIS  comes  from  open 
literature,  access  to  the  system  GE  developed  is  largely  limited  to  its 
own  staff  (Westbrook,  1981) .  Industry-operated  centers  also  restrict 
output  to  members.  In  some  cases,  they  limit  access  to  project  results 
to  firms  supplying  support  for  that  project.  Release  to  other  members 
or  to  the  public  may  be  delayed,  if  forthcoming  at  all. 

The  present  alternative:  Informal  contacts.  At  present,  then,  there 
are  major  obstacles  to  screening  data  for  quality  in  a  research  area 
outside  of  one  in  which  the  user  is  professionally  involved.  As  a 
result,  we  learned  that  most  users  rely  as  much  as  possible  on  collegial 
contacts.  "Knowing  who  to  call"  was  the  solution  to  problems  of 
locating  high  quality  research  literature  employed  most  frequently  by 
individuals  we  spoke  with  in  academic,  industrial  and  government  agency 
settings.  Peer  networks  are  maintained  both  informally  and  formally 
(e.g.,  through  organized  meetings  of  special  interest  groups).  Several 
users  told  us  that,  if  members  of  their  own  contact  network  could  not 
directly  recommend  a  sought-for  research  product,  they  could  usually 
refer  the  inquirer  to  another  knowledgeable  source.  In  addition,  one 
industrial  engineer  remarked  that,  if  there  were  any  outstandingly  good-- 
or  bad--work  in  his  field  of  endeavor,  the  news  would  travel  rapidly 
through  collegial  circles  and  he  would  be  certain  to  hear  of  it. 

However,  through  informal  contacts,  individuals  can  learn  only  what 
others  in  their  network  happen  to  know--and  all  individuals  are  faced 
with  too  large  a  quantity  of  information  to  screen.  Clearly,  other  more 
systematic  means  of  evaluation  are  desirable. 


USER-RESPONSIVENESS 
Status  and  Achievements 

Few  formal  information  transfer  systems  allow  easy  access  and 
specification  of  search  and  delivery  procedures  by  the  user.  Despite 
the  ubiquitous  availability  of  information  and  the  presence  of  online 
abstracting  and  indexing,  most  online  searchers  are  information 
intermediaries  like  librarians.  Nevertheless,  with  the  development  of 
more  flexible  software,  online  services  are  beginning  to  design  systems 
to  enable  the  user  to  obtain  data  more  readily  on  their  own.  So  far, 
these  efforts  are  rudimentary  and  not  widespread. 

Mechanisms  to  create  or  enhance  informal  information  channels  have 
had  varying  success.  Databases  identifying  researchers  engaged  in 
ongoing  research,  such  as  the  Smithsonian  Science  Information  Exchange, 
have  received  little  usage.  The  SSIE  was  recently  discontinued,  and  the 
Federal  Research  in  Progress  database  created  by  NTIS  as  a  successor  has 
sharply  curtailed  descriptions  and  coverage.  In  contrast,  ARPANET, 

CSNET,  and  other  interactive  messaging  systems  have  proved  more 
successful  within  their  clientele,  probably  because  they  are  patterned 
more  closely  after  self -generated  collegial  exchange.  Programs 
employing  link  agents  to  unite  users  with  information  sources  have  had 
varied  support  and  varied  success.  The  NASA  technology  transfer 
programs  have  produced  some  successes,  although  their  regional 
information  analysis  centers  have  declined  in  recent  years.  On  the 
other  hand,  university- industry  research  programs,  designed  to  put 
knowledge  producers  and  users  into  direct  contact  with  one  another, 
establish  more  participatory  information  transfer  and  have  been 
expanded . 

Participants  in  our  discussions  indicated  that  federal  information 
services  often  failed  to  aggressively  market  their  wares  to  the  user 
community.  However,  measuring  the  success  of  formal  information 
channels  is  difficult.  Many  government  databases  rely  on  commercial 
online  services,  for  example,  to  provide  computerized  access. 

Information  on  customer  usage  is  thus  proprietary.  One  survey  did  show 
that  69  percent  of  defense  industry  engineers  surveyed  had  either  never 
heard  of  the  Department  of  Defense's  information  analysis  centers  or  did 
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not  know  what  they  did  (Corridore,  1976).  Seminars  and  workshops  may  be 
used  by  agencies  to  promote  awareness  of  key  technologies  and  new  tools. 
Some  programs  to  improve  curricula  in  higher  education  perform  a  similar 
function.  Several  private-  and  public-sector  information  managers 
indicated  that  major  changes  in  scientific  and  engineering  curricula 
should  be  made  to  train  new  generations  to  use  the  new  information 
access  tools. 

Users  may  attempt  to  become  more  familiar  with  online  systems  on 
their  own  if  they  believe  the  information  they  acquire  by  that  means  is 
worth  the  trouble  it  takes.  For  example,  in  recent  years,  some 
scientific  journals  have  been  made  available  as  machine-readable  data 
files.  Many  of  them  are  now  beginning  to  go  online  with  commercial 
services  such  as  BRS .  Full-text  searching  offers  access  to  articles  at 
the  level  of  specific  information  needs,  although  hampered  substantially 
by  the  current  omission  of  graphs,  figures,  and  other  non- alphanumeric 
information.  Advanced  online  information  services,  such  as  CAS  Online 
or  Questel/DARC,  are  beginning  to  offer  graphic  retrieval  capabilities 
for  users  with  appropriate  terminals.  The  chemical  information  systems 
lead  the  way  in  this  area,  but  graphic  full-text  systems  for  physics, 
electronics,  and  patents  are  in  the  planning  stages.  An  account  of  the 
development  of  federal  policies  and  practices  aimed  at  making  R&D 
information  products  more  user-responsive  is  presented  in  Appendix  B. 

Problems 

Like  quality  assurance,  user-responsiveness  issues  are  related  to 
information  availability.  Rapid  expansion  of  the  scientific  and 
technical  information  base  has  dramatically  increased  the  difficulty  of 
locating  problem-relevant  research  results  and  translating  them  into 
improved  products  or  processes.  As  Bozeman  and  Kim  (1981)  point  out, 

"it  cannot  be  assumed  that  scientific  knowledge  will  flow  into  industry 
at  a  rate  commensurate  with  its  utility"  (cf.  Bikson  et  al.,  1981).  Key 
obstacles  to  promoting  user-responsiveness  include  interdisciplinary 
barriers,  communications  gaps  between  information  producers  and  users, 
and  lack  of  interactive  user  guidance  in  the  information  transfer 
process.  Consequently,  contact  networks  and  interpersonal  interactions 
become  the  user's  major  tools  for  acquiring  research  knowledge. 
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Interdisciplinary  barriers.  Putting  research  knowledge  to  use 
frequently  requires  transferring  it  to  a  context  quite  different  from 
its  context  of  origin.  However,  scientific  and  technical  information  is 
organized  along  traditional  disciplinary  lines,  as  are  subject  matter 
indexes,  abstracts,  and  key  words.  Thus,  users  find  focused 
multidisciplinary  retrieval  extremely  difficult  (Adam,  1975;  Allen, 

1969).  Moreover,  this  problem  has  grown  increasingly  important  as 
disciplinary  boundaries  have  become  blurred  (Fjalestad,  1983;  Tornatzky 
et  al.,  1983).  Our  discussions  produced  several  examples  of  literature 
searches  that  became  excessively  cumbersome  and  unmanageable  because  of 
the  need  for  multidisciplinary  access.  These  searches  included  work  on 
manpower  projections,  international  industrial  competition,  and 
technology  transfer  itself.  Difficulties  in  obtaining  information  are 
exacerbated  when  different  methodologies  as  well  as  different  constructs 
are  united  in  one  research  project,  as  happens  in  hybrid  areas  such  as 
biotechnology,  environmental  impact  assessment,  and  the  like  (Lide, 

1980;  Bozeman  and  Blankenship,  1979).  In  short,  formal  information 
transfer  procedures  are  not  responsive  to  users'  needs  when  the  user  s 
environment  is  not  well  aligned  with  standard  disciplinary  taxonomies. 

Formal  dissemination  mechanisms  are  particularly  ineffective  for 
transferring  results  from  research  to  application,  since  they  do  not 
provide  for  the  organization  of  information  according  to  problem 
relevance.  Rather,  as  we  have  noted,  secondary  sources  yield  pointers 
to  the  scientific  subject  matter  of  research  and  give  no  indication  of 
its  potential  instrumental  value.  As  one  engineering  researcher  put  it: 
"Science  is  universal,  but  technology  is  local"  (cf.  Tornatzky  et  al., 
1983;  Bikson,  1980).  Blurring  of  the  distinction  between  "basic"  and 
"applied"  research  has  not  improved  the  problem-relevance  of  formal 
dissemination  mechanisms  (Fjalestad,  1983;  National  Academy  of 
Engineering,  1981). 

Two  sorts  of  targeting  efforts  have  been  undertaken  to  increase  the 
likelihood  that  research  results  will  reach  users  who  need  and  will 
apply  them.  One  is  to  identify  potential  users  of  the  results  of  a 
research  project  and  distribute  its  information  products  directly  and 
promptly  to  them  (e.g.,  Glaser,  1980;  Rich,  1981).  This  approach  has, 
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with  some  exceptions,  not  been  notably  successful.  "You  can't  expect 
solutions  to  go  looking  inside  organizations  for  problems,"  commented  a 
discussant.  However,  others  believed  that  a  more  careful  effort  to 
target  opinion  leaders  in  potential  user  clienteles  might  help, 
following  the  strategy  of  mass  media  in  segmenting  the  market  and 
stimulating  product  demand  accordingly  (cf.  Wright,  1969). 

The  second  type  of  targeting  effort  has  been  to  define  areas  of 
national  need  and  fund  problem-oriented  research  in  those  domains,  on 
the  assumption  that  the  results  will  be  readily  applied  when  available 
(e.g.,  NSF's  former  RANN  program).  This  approach  too  has  had  only  mixed 
success.  Those  with  whom  we  spoke  generally  believed  that 
overmanagement  of  research  from  an  applied  perspective  would  impair 
scientific  advance;  in  addition,  they  believed  that  users  themselves 
were  much  better  able  to  define  priorities  for  applied  research  than 
were  government  agencies  (cf.  National  Academy  of  Engineering,  1981). 

In  sum,  formal  one-way  "supply-side"  transfer  procedures  do  not 
seem  to  be  responsive  to  the  user  context.  Rather,  these  efforts  appear 
to  start  with  an  information  system  into  which  they  later  try  to 
retrofit  users'  requirements  (Adam,  1975).  Indeed,  many  discussants 
believed  that  dissemination  activities  were  afterthoughts,  undertaken 
without  serious  commitment  by  federal  agencies  whose  primary  concerns 
were  with  research  production  and  not  with  knowledge  transfer.  Further, 
70  percent  of  federally  supported  R&D  is  carried  out  in  private  or  local 
public  institutions.  The  effective  coupling  of  these  undertakings  with 
specific  user  needs  would,  under  any  circumstances,  require  considerable 
oversight  and  management.  These  qualities  are  likely  to  be  difficult  to 
secure,  judging  from  the  lack  of  coordination  of  science  information 
policy  across  government  agencies  (Branscomb,  1980;  Bozeman  and  Kim, 
1981) . 

The  producer- user  gap.  While  dissemination  policies  and  practices 
have  chiefly  employed  one-way  source-to-user  transmission,  the  consensus 
of  findings  from  empirical  research  is  that  interactive,  two-way 
communications  are  required  for  effective  information  transfer  (cf. 
Tornatzky  et  al.,  1983;  Beyer  and  Trice,  1982;  Gerstenfeld  and  Berger, 
1980).  That  is,  much  of  what  has  been  learned  about  effective 
information  transfer  has  not  been  incorporated  into  federally  supported 
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dissemination  activities.  An  often-cited  impediment  to  the  development 
of  a  two-way  exchange  are  the  well-established  "cultural  differences" 
between  information  producers  and  potential  users.  These  groups  have 
different  values,  norms,  assumptions,  focal  interests,  and  communication 
networks;  they  belong  to  nonoverlapping  collegial  circles  (Beyer  and 
Trice,  1982;  Glaser,  1981;  Bikson,  1980).  Nevertheless,  some 
dissemination  procedures  have  supported  interactive  exchange  using 
persons  as  knowledge  transfer  media. 

One  set  of  practices  entails  putting  active  intermediaries  into  the 
knowledge  transfer  process.  In  some  instances,  the  intermediaries 
represent  the  information  supply  side;  affiliated  with  funding  agencies 
or  research-producing  institutions,  these  individuals  are  expected  to 
serve  as  "knowledge  brokers"  or  "linking  agents"  (e.g.,  McGowan  and 
Loveless,  1981;  Goldhor  and  Lund,  1980).  Intermediaries  affiliated  with 
user  organizations,  in  contrast,  act  as  "technological  entrepreneurs"  or 
"gatekeepers"  (e.g.,  Bozeman  and  Cole,  1982;  National  Academy  of 
Engineering,  1981;  Allen,  1977).  Individuals  in  these  roles  filter 
information  for  relevance,  translate  it  into  usable  form,  and  give 
specific  problem-oriented  feedback.  Identification  of  appropriate 
individuals  and  support  of  their  functions  has  been  attempted  on  a 
limited  basis  (e.g.,  Glaser,  1978),  and  most  individuals  we  talked  with 
thought  such  practices  should  be  more  systematically  encouraged. 
Empirical  findings  on  the  effectiveness  of  these  intermediary  roles  are, 
however,  sparse  and  inconclusive  (Beyer  and  Trice,  1982;  Rich,  1981). 
Their  impact  is  likely  to  be  strongly  conditional  on  specific 
institutional  contexts,  and  they  may  be  costly  to  maintain. 

A  second  set  of  practices  involves  putting  research  producers  and 
users  into  more  direct  contact  with  each  other.  This  approach  seems 
more  promising  because  it  makes  information  exchange  a  function  of  the 
regular  activities  of  both  producer  and  user  groups  and  because,  once 
initiated,  it  does  not  have  to  depend  on  special  funding  for  its 
maintenance.  A  long-recognized  and  highly  successful  person-to-person 
transfer  of  research  knowledge  occurs  when  graduate  students  obtain 
employment  in  applied  science  and  technology  settings.  The  main  problem 
with  support  of  graduate  education  as  a  transfer  mechanism  is  that  it 
takes  too  long  (cf.  NASA,  1966).  For  more  timely  dissemination,  other 
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types  of  manpower  exchange  between  information  producer  and  user 
communities  are  valuable.  Most  of  our  discussants  thought  that  one- 
shot  site  visits  and  joint  conference  attendance  were  desirable  but  less 
productive  than  the  longer-term  interactions  that  result  from  employee 
exchange  programs  or  from  consultative  arrangements,  although  the  former 
may  promote  the  latter  (cf.  Beyer  and  Trice,  1982;  Arthur  D.  Little, 

1979;  Mesthene,  1972). 

Other  major  avenues  for  direct  interaction  between  information 
producers  and  users  are  collaborative  research  projects  and  more 
comprehensive  cooperative  research  centers  (Eveland  et  al.,  1982; 
Tornatzky  et  al.,  1982;  National  Science  Board,  1982;  Baer,  1980).  The 
cultural  barriers  cited  above  tend  to  present  initial  obstacles  to  the 
formation  of  such  alliances.  For  example,  a  vice-president  of  a  product 
development  laboratory  told  us  that  "academics  have  no  idea  what  might 
be  useful  to  industry";  their  orientation  is  "too  abstract,"  their 
timeframe  is  "unrealistic,"  and  their  attitude  is  "contrary  to  transfer" 
and  "holier  than  thou"  toward  applied  scientists.  On  the  other  hand,  a 
university  engineering  professor  remarked  that  industry  representatives 
have  a  "very  narrow  focus";  they  are  "so  practical  and  so  development- 
oriented"  that  they  "lack  the  perspective  for  looking  at  the  long-term 
significance  of  more  general  research."  In  addition,  participation  of 
private  for-profit  firms  with  public  or  not-for-profit  institutions  in 
joint  scientific  and  technical  activities  raises  questions  about  the 
protection  of  proprietary  data  as  well  as  the  patenting  and  licensing  of 
resultant  products. 

The  many  successful  examples  of  ongoing  collaborative  research 
indicate  that  such  barriers  can  be  overcome,  and  representatives  of 
industries  and  universities  concurred  in  urging  the  strengthening  of 
these  efforts.  Advantages  are,  first,  that  they  give  rise  to  "generic" 
research  that  is  intermediate  between  basic  science  and  product-specific 
development,  promoting  basic-to-applied  transfer.  Second,  continued 
interaction  permits  information  to  be  integrated  and  focused  in  response 
to  user  needs,  facilitating  timely  application.  Third,  these  efforts 
give  more  leverage  to  federal  research  funds  by  stimulating  industrial 
support  (Slaughter  1981).  Finally,  they  have  done  much  to  alleviate  the 
industry-university  culture  gap;  "a  significant  improvement  in  these 
relationships  appears  to  be  underway"  (Fowler,  1984). 
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Lack  of  user  control.  As  we  have  seen,  a  shift  from  supply-side 
emphases  to  more  interactive  models  of  dissemination  is  required  when 
the  policy  objective  moves  from  information  availability  to  information 
usability.  The  logical  extension  of  this  trend  is  the  development  of 
policies  that  enable  users  themselves  to  have  greater  control  over 
knowledge  transfer  processes  for  purposes  of  achieving  product  or 
process  improvements.  Since  the  classic  study  by  Coch  and  French 
(1948),  it  has  been  acknowledged  that  participation  in  an  information 
process  significantly  increases  the  likelihood  that  participants  will 
act  on  the  results .  Three  types  of  approaches  to  shifting  the  knowledge 
transfer  initiative  to  the  user  side  have  been  attempted: 

•  Limited,  experimental  attempts  have  been  made  to  adapt 
information  technology  so  that  users  themselves  can  command 
information  resources.  Briefly  reviewed  in  Wigington  (1982), 
these  approaches  include  software  for  permitting  users  to 
define  information  searches  around  their  own  needs  and  to 
retrieve  and  manipulate  data.  They  also  include  access  to 
electronic  contact  networks  that  facilitate  information 
resource  sharing  among  users.  According  to  Bozeman  and 
Blankenship  (1979),  automated  systems  must  be  user-guided  or 
they  will  repeat  all  the  known  problems  of  other  dissemination 
procedures.  Most  of  our  discussants  believed  they  could  be 
taking  better  advantage  of  advanced  information  technology  than 
they  were  at  present  (cf.  Fjalestad,  1983;  Engelbart,  1975; 
Adam,  1975;  NASA,  1966). 

•  Another  approach  is  to  encourage  interaction  within  user 
communities  (e.g.,  Fusfeld,  1979;  Brigham,  1975).  Such 
exchange  has  been  shown  to  lead  to  innovation  and  to 
application  (Bozeman  and  Bozeman,  1981;  Gerstenfeld  and  Berger, 
1980).  According  to  Tornatzky  et  al.  (1983),  recognition  of 
networked  or  "peer-matched"  models  of  scientific  communication 
as  alternatives  to  the  center-periphery  or  clearinghouse  models 
should  help  promote  direct  interorganizational  interaction. 
However,  there  are  serious  institutional  barriers  to  resource 
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sharing  and  research  collaboration  among  industrial  users. 

Said  one  representative  of  a  collaborative  program,  "A  major 
impediment  is  that  all  industries  have  to  compete,"  which  means 
they  have  to  "hoard  results"  instead  of  cooperate.  According 
to  Rich  (1981),  institutional  boundaries  and  competitiveness 
also  impair  resource  sharing  among  public  agency  users. 

•  A  third  approach  is  to  provide  tax  incentives  and  regulatory 
reforms  to  strengthen  private-sector  participation  in  R&D  and 
promote  utilization.  For  example,  the  Economic  Recovery  Act  of 
1981  awards  tax  credits  to  for-profit  firms  that  support  basic 
research.  Similarly,  recent  changes  in  patent  regulations 
permit  some  exclusive  license  rights  to  firms  that  have 
developed  innovative  products  on  the  basis  of  federally  funded 
research  results.  The  impact  of  these  policies  on  knowledge 
transfer  has  not  yet  been  systematically  evaluated.  However, 
discussants  believed  that  counter- incentives  (such  as  antitrust 
laws  and  unrecoverable  innovation  costs)  still  outweigh 
positive  factors  (cf.  Pavitt,  1979;  Mesthene,  1972). 

Lide  (1981)  and  others  believe  that  effective  strategies  for 
matching  information  resources  to  information  needs  deserve  immediate 
attention.  Such  strategies  take  into  account  the  fact  that  effective 
user-oriented  transfer  is  not  just  an  information-screening  issue  but  is 
also  a  behavioral,  institutional,  and  economic  issue.  Without  these 
strategies,  it  is  likely  that  "information  will  continue  to  be  collected 
and  absorbed  at  an  exponential  rate  while  the  ability  to  understand  and 
manage  it  will  diminish"  (McGowan  and  Loveless,  1981). 
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III.  POLICY  OPTIONS  AND  THEIR  MERITS 


Review  of  past  and  present  dissemination  policies  and  practices 
clearly  establishes  a  need  for  more  interactive,  user-guided  information 
transfer.  Passive  access  to  voluminous  stores  of  textual  and  numeric 
data  does  little  to  promote  timely,  selective  retrieval  and  application 
of  high-quality  R&D  results.  Therefore,  in  the  options  we  describe  for 
improving  information  transfer,  the  emphasis  is  on  increasing  the  user- 
responsiveness  and  selectivity  of  existing  formal  dissemination  systems 
and  coupling  them  more  closely  with  informal  systems. 

We  organize  our  discussion  of  policy  options  into  three  groups-- 
techno logy -intensive  options,  supply-side  options,  and  user-focused 
options.  Of  course,  the  division  is  somewhat  arbitrary,  and  specific 
alternatives  are  interrelated  in  that  the  merits  of  some  will  depend  on 
whether  others  are  adopted.  In  each  category,  we  give  greatest 
attention  to  those  we  judge  to  be  the  most  promising  alternatives. 
Further,  we  emphasize  the  needs  of  nonacademic  users  of  information, 
because  our  evaluation  indicates  they  are  at  greatest  disadvantage  under 
current  dissemination  procedures. 

TECHNOLOGY-INTENSIVE  OPTIONS 

By  technology-intensive  options,  we  mean  policy  alternatives  that 
take  advantage  of  the  special  properties  of  computer  and  communications 
technology  for  the  transfer  of  scientific  and  technical  information. 
Technology-intensive  options  serve  two  major  objectives:  putting  users 
into  more  direct  contact  with  stored  information,  and  putting  users  into 
more  direct  contact  with  other  parties  to  the  scientific  exchange. 

The  most  important  features  of  electronic  media  for  this  purpose 
are  the  flexibility,  manipulability ,  timeliness,  and  interactivity  that 
they  can  contribute  to  information  transfer.  Although  the  publications 
and  information  services  industries  rely  increasingly  on  electronic 
technology  for  internal  operations,  we  focus  here  on  its  potential  for 
enhancing  the  participation  of  end  users  in  the  dissemination  process. 
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Interactive  Networks.  Policies  could  be  designed  to  promote  and 
support  the  development  of  computer-based  connected  networks  that  will 
allow  interaction  among  participants  in  the  scientific  communication 
process.  For  this  purpose,  the  federal  government  might  fund 
exploratory  efforts  to  determine  how  electronic  technology  can  best  be 
deployed  to  extend  the  known  benefits  of  informal  interaction.  These 
experimental  efforts  could  take  a  variety  of  directions,  as  we  suggest 
below.  We  believe  that  the  seeds  of  substantive  advance  may  well  be 
found  in  this  arena. 

•  Electronic  message  exchange  systems  can  potentially  generate  ad 
hoc,  self -configuring  groups.  Interest  groups  of  this  type  can 
carry  out  functions  of  informal  systems  such  as  alerting 
members  to  research  projects  initiated,  broadcasting 
information  needs,  exchanging  quality  judgments,  passing  on 
significant  new  findings,  conferring  about  applications,  and 
the  like  (cf.  Cronin,  1982).  Current  ARPANET  groups  function 
successfully  in  this  model. 

•  Communication  networks  can  also  be  used  for  more  extended 
information  resource  sharing,  such  as  exchanging  primary  data 
or  preprints,  forwarding  documents  or  references  to  them,  and 
so  on.  Head  notes  from  other  network  members  would  help  users 
filter  and  select  from  available  online  materials.  In  this 
fashion,  users  might  develop  their  own  dynamically  updated  and 
customized  information  resources  (cf.  Engelbart,  1975). 

•  A  community  of  users  participating  in  such  a  network  over  time 
would  generate  valuable  by-products.  For  example,  user-profile- 
based  mechanisms  for  helping  users  locate  relevant  information 
products  can  be  made  self-improving;  or,  shared  evaluations  of 
information  products  can  be  used  to  derive  parameter  weightings 
helpful  in  qualitative  filtering  of  information  products. 

•  Project-based  networks  could  be  initiated  to  put  representative 
potential  users  in  contact  with  researchers  throughout  the 
course  of  a  project.  Feedback  to  knowledge  producers  would 
help  insure  the  usability  of  the  results,  while  users  would 
have  timely  access  to  project  findings. 
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•  Finally,  such  networks  could  be  interconnected  to  yield  a 
multiplicity  of  coupled  systems  that  would  constitute  a 
national  decentralized  information  sharing  capability  (NASA, 
1966).  That  is,  while  allowing  for  the  formation  of  specific 
groups,  computer-based  communication  systems  also  provide  for 
the  possibility  of  one  "electronic  college." 

For  exploratory  research  to  test  the  effectiveness  of  policy 
options  of  the  sort  described  above,  implementation  is  the  primary 
consideration.  A  real  opportunity  for  rapid  advancement  in  information 
dissemination  and  utilization  may  lie  in  stimulating  and  supporting  the 
use  of  interactive  networks.  Many  of  our  discussants  suggested 
innovative  R&D  efforts  in  this  area.  We  believe  it  would  be  productive 
to  explore  the  feasibility,  perhaps  on  a  pilot  project  basis,  of  a 
dissemination  policy  option  which  would  combine  the  following 
properties : 

•  Establishing  a  National  Science  Network  (NSN) .  This  network 
could  use  the  identical  technology  to  ARPANET  and  thus 
capitalize  on  the  15  years  or  so  of  research  which  underlies 
its  success.  Many  of  the  information  dissemination,  access, 
and  sharing  techniques  already  well  established  could  be 
applied  immediately  to  the  U.S.  scientific  community  at  large. 

•  Mandating  access  to  the  NSN,  perhaps  as  a  part  of  conducting 
NSF-sponsored  research.  NSF  could  use  the  network  for  proposal 
submission,  award  announcement,  collection  of  results,  and 
reviews.  The  public  at  large  could  be  granted  access  to  the 
network  on  a  pay  as  you  go  basis. 

•  Encouraging  information  entrepreneurs  to  use  the  network  and 
data  bases  connected  thereto  to  provide  worthwhile  and 
profitable  information  dissemination  services. 

These  kinds  of  policy  initiatives  take  advantage  of  electronic 
technology  to  establish  direct,  timely,  and  interactive  contact  among 
participants  in  the  R&D  dissemination  process.  They  couple  the  power  of 
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computer  systems  and  human  communications  processes  to  provide  members 
of  the  scientific  community-information  users  and  producers  - -with 
access  to  each  other.  Ongoing  self -generated  exchange  would  be  expected 
to  promote  knowledge  transfer  and  enhance  innovative  utilization. 

Because  these  options  make  use  of  extant  technology  and  known 
procedures,  they  avoid  heavy  technical  development  costs.  Further,  once 
shown  to  be  of  value,  they  are  readily  generalized.  Because  they 
represent  a  marked  departure  from  present  practice,  such  initiatives  are 
exploratory  and  not  without  risk.  On  the  other  hand,  in  contrast  with 
marginal  improvements  in  conventional  approaches,  they  hold  the 
potential  for  significant  new  benefits.  We  believe  the  technology 
infrastructure  for  such  activities  is  fairly  well  developed.  Most 
researchers  and  many  government  and  industry  users  have  access  to 
computers  and  communications  systems.  However,  small  businesses,  local 
government  agencies,  and  other  users  who  lack  access  to  computer 
resources  might  require  special  assistance. 

User-guided  search  and  retrieval.  Policies  could  be  designed  to 
support  the  exploration  and  development  of  more  flexible  and  powerful 
user-guided  methods  for  accessing  and  retrieving  highly  structured 
textual  and  numeric  databases.  For  this  purpose,  the  federal  government 
might  fund  adaptations  of  high-level  languages  and  knowledge-based 
systems  for  intelligent  location  and  use  of  scientific  and  technical 
information . 

•  A  mechanism  could  be  provided  to  enable  users  to  express 
requests,  define  ways  of  examining  a  database,  browse,  make 
heuristic  searches,  carry  out  inferences,  and  the  like.  Many 
of  the  participants  in  our  discussions  regard  supporting  these 
capabilities  for  users  as  far  more  effective  than  trying  to 
specify  in  advance  fixed  sets  of  search  criteria  or  to  explain 
to  someone  else  what  kinds  of  information  might  be  desired. 

•  A  knowledge  base  could  be  constructed  to  encompass  a  set  of 
bibliographic  or  numeric  databases  relevant  to  the  interests  of 
a  user  group(s).  The  knowledge  base  can  serve  as  an 
intelligent  intermediary  between  the  end  user  and  stored 
information,  "knowing"  which  databases  should  be  searched  in 
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response  to  a  given  request  and  which  search  procedures  might 
be  employed.  Several  individuals  recommended  applying  such 
artificial  intelligence  approaches  to  the  problem  of 
selective,  user-responsive  information  retrieval. 

•  Systems  can  be  developed  to  permit  users  to  manipulate  the  data 
retrieved  in  a  search.  Whether  it  is  textual  or  numeric,  the 
information  output  resulting  from  selective  user-guided 
searches  could  be  made  available  for  subsequent  processing  by 
users  and  direct  incorporation  into  their  work. 

•  For  numeric  databases,  coupling  of  search  and  retrieval 
processes  with  manipulation  and  analysis  procedures  would  allow 
for  comparative  examination  and  evaluation  of  stored 
information  either  by  an  information  service  provider  or  an  end 
user.  Data  cleaning  programs  have  been  developed  over  some 
databases  for  this  purpose;  they  could  be  made  more  generally 
available . 

Efforts  to  provide  user-guided  search  and  retrieval  mechanisms  and 
intelligent  interfaces  to  multiple  databases  would  require  the  funding 
of  research  and  development  projects.  These  capabilities  are  beyond  the 
current  state  of  the  art,  although  it  may  be  possible  to  adapt  existing 
technology  to  these  purposes.  For  example,  a  powerful  and  flexible 
language  would  have  to  be  adapted  to  permit  nontechnical  users  to  query 
a  database  and  guide  a  search.  Textual  or  numeric  data  would  have  to  be 
consistently  structured  and  formatted  for  such  treatment,  ideally  linked 
to  a  knowledge  base  that  could  provide  the  mechanism  for  intelligent 
computer-aided  searching.  However,  projects  that  apply  advances  in 
artificial  intelligence  and  expert  systems  research  to  problems  of 
knowledge  transfer  were  seen  by  our  discussants  as  having  significant 
long-term  potential  for  improving  the  integration,  evaluation,  and 
utilization  of  growing  stores  of  scientific  and  technical  information. 

Other  Options.  Other  options,  suggested  both  in  the  literature  and 
in  discussions,  for  taking  better  advantage  of  computer  and 
communications  technology  in  the  dissemination  process  primarily 
concerned  electronic  publication.  That  is,  many  believe  that  the 
federal  government  could  take  steps  now  to  encourage  and  support  the 
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electronic  storage,  distribution,  and  consumption  of  primary  research 
and  development  products  as  well  as  secondary  products  such  as  texts, 
reviews,  and  handbooks.  Although  electronically  captured  information 
can  readily  be  printed,  going  from  print  copy  to  electronic  copy  may  be 
costly  and  time-consuming,  and  can  introduce  new  errors.  Consequently, 
insofar  as  possible,  it  would  be  desirable  to  have  research  reports  and 
other  deliverables  submitted  in  electronically  accessible  form.  In 
particular,  any  products  that  require  regular  updating  (e.g.,  handbooks 
and  compilations)  could  be  maintained  online.  Equally  important, 
information  products  that  potentially  bear  on  the  same  problem- -whether 
textual  or  numeric- -could  be  retrievable  through  an  integrated  access 
mechanism  whether  they  are  in  the  same  or  different  databases. 

As  a  corollary,  the  federal  government  could  consider  exploring, 
designing,  and  promulgating  standards  for  consistency  and  compatibility 
in  computer  and  communications  systems  used  to  disseminate  scientific 
and  technical  information- -along  with  other  major  actors  including 
representatives  of  private  industry  and  international  scientific  bodies. 
Standardization  might  be  pursued  broadly  in  relation  to  all  aspects  of 
information  technology  that  affect  connectivity;  that  is, 
standardization  efforts  need  to  consider  not  only  the  components 
directly  involved  (e.g.,  storage  and  retrieval  mechanisms)  but  also  the 
body  of  lower  level  information  technology  standards  (e.g., 
communication  protocols)  on  which  they  necessarily  rest.  In 
consequence,  any  move  toward  standardization  would  confront  a  host  of 
unresolved  technical  issues.  However,  we  emphasize  that  the  unexplored 
value  of  electronic  technology  does  not  lie  merely  in  increasing  the 
efficiency  of  the  formal  publication  and  dissemination  system  as  it  now 
exists,  but  rather  in  dynamically  uniting  that  system  with  its  users. 

SUPPLY-SIDE  OPTIONS 

We  describe  below  some  policy  alternatives  that  attempt  to  improve 
dissemination  processes  from  the  information  supply  side;  they  could 
make  use  of  but  do  not  require  the  more  technology-intensive  options 
discussed  above. 
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Improving  information  organization.  Our  evaluation  of  current 
dissemination  procedures  established  the  need  to  improve  the  access 
structure  for  stored  information.  Indexes  and  abstracts  do  not  enable 
users  to  make  highly  selective  searches,  particularly  when  their 
information  needs  are  multidisciplinary  or  applied.  The  federal 
government  could  support  exploratory  efforts  to  improve  the  filtering 
capability  of  these  tools.  Recommendations  we  encountered  to  increase 
search  selectivity  included  the  following: 

•  Abstracts  themselves  could  provide  more  complete  research 
information,  especially  about  methodology  employed, 
availability  of  data,  and  expected  areas  of  application. 
Improving  the  contents  of  abstracts  on  a  nonretroactive  basis 
would  be  less  difficult  or  costly  to  implement.  Changes  in  key 
words  or  addition  of  new  information,  such  as  data  flags,  to 
extant  databases  would  be  much  more  expensive  and  time- 
consuming  . 

•  Information  can  be  organized  according  to  "naturally"  occurring 
clusters.  Making  use  of  citation  indexes  and  current 
bibliometric  techniques  (see  Appendix  B) ,  cocitations  could  be 
used  to  derive  an  interrelated  collection  of  scientific  and 
technical  information  ad  hoc-- in  response  to  individual  user 
requests  - -rather  than  by  specifying  keywords  from  a 
predetermined  list.  This  type  of  user-oriented  organization 
procedure  could  reduce  the  quantity  of  information  retrieved  in 
a  search  while  improving  its  relevance  and  quality,  as 
experimental  studies  have  shown  (Price,  1974). 

•  Abstracts  and  key  words  could  be  organized  "conceptually,"  with 
terms  from  different  disciplines  serving  as  pointers  to  common 
knowledge  domains.  This  type  of  organizational  effort  would 
require  creating  an  interface  capable  of  reconciling  key  words 
for  similar  subject  matters  in  different  databases  as  well  as 
joining  associated  concepts  from  different  disciplines  in  the 
same  or  different  databases.  Besides  helping  to  integrate 
diverse  literatures,  this  option  would  also  help  to  eliminate 
redundancy  in  searches  (cf.  Adam,  1975). 
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The  last  policy  option  would  require  developmental  research,  while 
the  others  could  be  implemented  with  extant  technology.  All  three  aim 
at  making  desired  information  easier  to  find.  Their  effect  on  the 
dissemination  process  would  depend  on  whether  the  increased 
discriminability  they  provide  motivates  users  to  take  better  advantage 
of  passive-access  avenues  to  growing  information  resources 

Data  quality.  Quality  assurance  involves  two  sorts  of  issues:  (1) 
Can  users  trust  numeric  data  collected  in  research  projects?  (2)  Can 
users  trust  the  conclusions  drawn  from  the  research?  The  options  below 
address  the  former  question.  Our  examination  of  current  policies  and 
practices  led  us  to  conclude  that  secondary  uses  of  numeric  data  could 
be  significantly  increased  if  data  quality  were  assured.  Moreover,  the 
importance  of  secondary  data  use  increases  as  funds  for  primary  data 
collection  become  scarce.  Recommendations  in  this  area  include  the 
following: 

•  In  a  number  of  research  fields  where  property  measurements  or 
observational/environmental  data  are  collected,  standards  for 
evaluation  exist  (e.g.,  CODATA).  Many  sources  recommend  the 
systematic  allocation  of  funds  for  evaluation  of  resulting  data 
of  this  type  when  grants  or  contracts  are  awarded.  It  has  been 
estimated  that  evaluation  would  require  5  percent  or  less  of 
the  cost  of  data  generation  (National  Academy  of  Sciences, 

1978).  The  federal  government  might  investigate  alternative 
mechanisms  for  accomplishing  data  evaluation: 

Information  analysis  centers  could  expand  the  scope  of 
their  data  evaluation  activities. 

Evaluation  efforts  could  be  flexibly  staffed  by  experts 
on  a  consultative  basis;  after  termination  of  a  primary 
research  project,  results  could  be  appraised  and 
integrated  with  existing  critically  evaluated  data 
sources . 

Given  integrated  online  datasets  and  access  to  data- 
cleaning  tools,  evaluation  could  be  carried  out  by 
users  on  an  as-needed  basis. 
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•  Statistical  data  (on  public  health,  economics,  demographics  and 
the  like)  still  require  the  development  and  promulgation  of 
criteria  for  quality  evaluation.  Agencies  that  fund 
collection  of  statistical  data  could  collaborate  in  defining 
evaluation  criteria.  Eventually,  statistical  data  evaluation 
could  be  handled  in  the  same  way  as  property  measurement  and 
observational /environmental  data . 

•  Finally,  several  sources  recommended  exploring  procedures  to 
integrate  textual  information  with  numeric  databases  for 
annotating  specific  problems  (e.g.,  measurement  difficulties, 
methodological  difficulties,  problematic  codes  and  the  like). 
Many  sources  (e.g.,  Robbin,  1981;  David,  1980)  mentioned  that 
lack  of  such  annotations  either  rendered  otherwise  adequate 
files  unusable  or  else  induced  major  delays.  Development 
research  would  be  required  to  act  on  this  option 

Integration  and  evaluation  of  numeric  data,  however  undertaken, 
involves  combining  information  from  many  sources  and  thus  entails  high 
costs,  in  part  because  of  inconsistent  formats,  file  structures,  and  the 
like.  Consequently,  such  tasks  would  be  aided  by  standardization  (see 
above).  To  reduce  initial  costs,  data  quality  assurance  could  be 
supported  first  in  areas  where  government  and  industry  have  identified 
needs  for  critically  evaluated  sources. 

Active  information  intermediaries.  Dissemination  literature 
indicates  that  information  is  best  transferred  by  a  human  agent  capable 
of  entering  into  a  dialog  with  users.  This  sort  of  exchange  is 
particularly  important  when  the  information  is  for  an  applied  problem. 
Further,  the  transfer  agent  needs  to  be  substantively  knowledgeable  in 
order  to  respond  to  the  questions  raised  by  potential  users.  There  are 
numerous  models  for  building  this  sort  of  function  into  the  activities 
of  knowledge  producers  (e.g.  Glaser,  1978).  Several  sources  have 
recommended  that,  near  their  completion,  research  projects  be  evaluated 
for  their  potential  application  to  either  government  or  industry. 
Agencies  could  provide  low  levels  of  funding  (or  explore  other 
incentives)  to  allow  representatives  of  promising  projects  to  act  as 
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knowledge  brokers  after  the  research  period.  In  this  capacity,  they 
could  provide  user-relevant  information  and  technical  assistance. 

The  cost  of  limited  support  for  information  intermediaries  is 
difficult  to  estimate  but  is  likely  to  be  a  major  factor  in  decisions 
about  their  use.  In  some  instances,  public  research  institutions  have 
been  willing  to  bear  the  expense  as  part  of  their  community  relations 
activities.  In  other  cases,  these  activities  are  seen  as  laying  the 
groundwork  either  for  further  collaborative  research  or  for  consultative 
user-compensated  arrangements. 

Other  options.  Other  supply-side  options  we  reviewed  relate  mainly 
to  coordinating  information  supply.  For  example,  some  have  mentioned 
the  possibility  of  preparing  and  maintaining  a  single,  complete, 
continuously  updated  directory  of  scientific  and  technical  information 
resources  (in  contrast  to  the  many  partial  listings  that  now  exist). 

Such  a  register  could  tell  how  to  find  out  about  information  services 
and  products;  it  could  list  evaluated  data  sources  and  provide  a  guide 
to  technology  transfer  efforts  (cf.  National  Academy  of  Engineering, 
1981).  The  need  for  agency-wide  coordination  has  also  been  remarked; 
most  individuals  we  talked  with  have  had  frustrating  and  unsuccessful 
experiences  trying  to  learn  about  the  existence,  location,  or  content  of 
agency  reports.  These  problems  might  be  alleviated  if,  within  each  R&D 
funding  agency,  an  office  were  established  with  responsibility  and 
authority  for  agency-wide  tracking  and  management  of  information 
products.  Valuable  insights  about  the  feasibility  of  any  of  the  above 
options  might  be  derived  from  a  study  of  what  real  incentives  R&D 
funding  agencies  have  to  devise  and  carry  out  effective  dissemination 
programs . 

USER-FOCUSED  OPTIONS 

Not  much  attention  has  been  paid  to  policy  options  that  empower 
users  to  play  a  stronger  role  in  the  transfer  of  scientific  and 
technical  information.  However,  our  review  suggests  that  user-focused 
alternatives  are  quite  promising.  But  because  users  comprise  such 
heterogeneous  groups  and  because  there  is  no  national  policy  history  on 
which  to  build  in  this  area,  it  is  difficult  to  provide  confident 
recommendations.  In  fact,  some  discussion  participants  suggested  the 
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best  first  step  might  be  to  convene  policy  brainstorming  sessions  with 
groups  of  potential  users  to  elicit  innovative  and  promising  options. 

We  believe  that  idea  has  considerable  merit,  given  the  quality  of  the 
appraisals  and  recommendations  we  elicited  in  the  small  number  of 
discussions  we  conducted  during  this  project.  In  the  meantime,  we  offer 
the  following  options.  They  could  be  enhanced  by,  but  do  not  require, 
implementation  of  technology-intensive  options. 

Skill  building.  Discussion  comments  indicated  that  the  majority  of 
potential  information  users--even  those  in  large  organizations--either 
are  unaware  of  information  resources  in  their  field  or  do  not  know  how 
to  use  them.  In  particular,  it  was  suggested  that,  if  users  are  to  have 
a  more  active  role  in  information  search  and  retrieval,  it  is  important 
that  they  understand  types  of  information  products  and  services, 
information  systems  logic  and  characteristics,  and  information  resource 
management  (cf.  Beyer  and  Trice,  1982;  McGowan  and  Loveless,  1981; 
Bearman,  1981;  National  Academy  of  Engineering,  1981;  Garfield,  1979). 

We  refer  here  to  how  information  is  organized  and  accessed  in  general, 
not  to  protocols,  keywords  or  other  system-specific  properties. 

•  The  federal  government  might  support  institutions  of  higher 
education  in  developing  curricula  related  to  effective  use  of 
information  systems  by  people  other  than  information 
professionals.  Future  users  would  then  be  able  to  take  better 
advantage  of  advanced  information  tools  (Surprenant,  1982).  At 
the  same  time,  it  has  been  suggested  that  short  courses  or 
workshops  be  developed  in  this  area  for  current  users  employed 
in  industry  and  government. 

•  Second,  curricula  could  be  developed  to  fill  an  identified 
manpower  need  for  individuals  who  are  substantively 
knowledgeable  in  a  field  and  who  have  expertise  in  computer- 
based  information  systems.  This  new  breed  of  professionals 
would  be  capable  of  matching  users  to  information  resources  by 
virtue  of  being  (a)  conversant  with  the  scientific  and 
technical  domain,  so  as  to  understand  users'  information  needs; 
and  (b)  conversant  with  the  computerized  environment,  so  as  to 
translate  those  needs  into  information  actions. 
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•  Third,  it  was  suggested  by  a  number  of  discussants  that  special 
assistance  in  using  information  resources  be  made  available  to 
small  businesses  and  local  government  agencies.  Institutions 
without  slack  resources  may  be  unable  to  take  advantage  of 
nonproprietary  scientific  and  technical  information.  Ways  of 
supporting  access  to  computerized  information  services  and 
products  could  be  explored.  Further,  agency-based 
"consultants"  could  be  made  available  to  help  users  carry  out 
computer-based  search  and  retrieval. 

Stimulating  and  supporting  the  development  of  courses  related  to 
information  systems  use  would  fall  within  existing  programs  for  funding 
higher  education.  Once  in  place,  the  courses  could  be  maintained  by  the 
institutions  offering  them.  On  the  other  hand,  the  last  option  raises  a 
number  of  issues  about  the  fair  allocation  of  costs.  In  view  of  those 
difficulties,  we  give  it  a  lower  priority. 

User-driven  dissemination.  Empirical  research  consistently  finds 
that  when  information  transfer  actively  engages  the  users,  it  is  likely 
be  more  successful  (e.g.,  Tornatzky  et  al.,  1983;  Bikson,  1980).  As 
Bozeman  and  Blankenship  (1979)  note,  it  is  more  effective  to  build 
information  systems  around  user  preferences  than  to  try  to  build  user 
support  for  formal  systems  that  do  not  reflect  their  preferences. 

Several  policy  options  have  a  user-driven  orientation: 

•  A  number  of  research  projects  have  been  conducted  (e.g., 

Glaser,  1980)  that  relied  heavily  on  feedback  from  advisory 
committees  composed  of  potential  users.  As  a  result,  research 
products  reflected  user  needs  and  were  more  readily,  promptly, 
and  widely  applied.  It  has  been  suggested  that,  in 
applications  for  grants  or  contracts,  researchers  be  required 
to  identify  potential  user  groups .  A  condition  of  awards  could 
be  the  naming  of  a  project  advisory  board  comprising 
representative  stakeholders  whose  function  would  be  to  guide 
instrumental ly  relevant  aspects  of  the  research,  review  final 
products,  and  assist  in  further  dissemination. 
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•  Second,  it  has  been  recommended  (cf.  Bozeman  and  Cole,  1982) 
that  efforts  be  made  to  identify  or  encourage  "gatekeepers"  or 
"technological  entrepreneurs"  within  user  settings.  These  are 
individuals  who  acquire  cutting-edge  scientific  and  technical 
information  more  rapidly  than  their  peers  and  translate  and 
diffuse  it  throughout  their  organizations.  As  Bozeman  and 
Cole  (1982)  point  out,  identifying  such  persons  in  advance  is 
difficult;  however,  methods  for  rewarding  them  should  be  much 
less  difficult  to  devise  and  could  be  explored. 

•  Third,  it  has  been  recommended  that  the  development  of  informal 
collegial  interactions  within  and  between  user  communities  be 
encouraged.  Boundary-spanning  communication  across 
organizations  has  consistently  led  to  knowledge  transfer  and 
innovation  (e.g.,  Gerstenfeld  and  Beyer,  1980).  However, 
informal  linkages  are  typically  less  well  developed  in 
nonacademic  communities,  even  though  nonacademic  users  are 
likely  to  make  more  use  of  verbally  communicated  information 
than  their  academic  counterparts.  Ways  of  facilitating  the 
development  of  user  contact  networks  could  be  explored. 

The  user-driven  dissemination  strategies  we  have  discussed  are 
difficult  to  appraise  because  very  little  is  known  about  how  user 
communities  are  organized  to  acquire  and  distribute  information.  This 
reflects  the  fact  that  both  research  and  policy  have  emphasized  the 
information  supply  perspective.  Thus,  while  there  are  examples  of 
successes  in  the  three  policy  option  areas  cited,  it  is  unclear  what 
would  be  required  to  implement  them  systematically.  On  the  other  hand, 
it  seems  evident  that  implementing  user-driven  policies  does  not  require 
federal  agencies  to  anticipate  and  select  key  problem  areas  or  key 
industries  as  dissemination  targets;  they  need  instead  to  increase  user 
involvement  in  setting  research  agendas  and  knowledge  transfer 
priorities  and  procedures. 

Collaborative  research.  We  have  learned  in  this  review  that  the 
nature  of  the  research  and  the  participants  in  the  research  process  have 
a  great  deal  to  do  with  subsequent  information  transfer  and  utilization. 
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Long  term  collaborative  research  efforts  that  address  a  multiplicity  of 
problem-related  issues  are  likely  to  yield  a  high-volume  transfer  of 
findings . 


Several  individuals  with  whom  we  spoke  during  this  study 
endorsed  the  option  of  increased  funding  for  "generic"  research 
intermediate  between  basic  science  and  product  development. 
Process  technologies  are  perhaps  the  clearest  example.  Such 
investigations  yield  findings  that  have  immediate  application 
value.  However,  the  building  of  research  teams  and  coherent 
bodies  of  results  should  be  viewed  as  a  long-term  effort. 
Short-term  narrowly-focused  one-shot  projects  were  regarded  as 
less  likely  to  generate  results  of  reliable  value. 

Discussants  as  well  as  literature  sources  concurred  in  urging 
continuation  and  expansion  of  support  for  collaborative 
industry/university  research.  Current  practices  have  done  much 
to  break  down  barriers  between  these  communities  (Fowler, 

1984) .  In  addition,  more  dual-function  research  and  production 
efforts  could  be  undertaken  (modeled,  for  example,  after  the 
submicron  structure  facility  at  Cornell,  which  is  both  a 
fabrication  resource  and  a  research  center  for  fabrication 
technologies).  Collaborative  research  does  much  to  leverage 
scarce  agency  funds  by  coupling  them  with  industrial  support. 

At  the  same  time,  joint  undertakings  provide  potential  users 
with  opportunities  for  input  and  interaction  throughout  the 
research  process  while  greatly  increasing  the  utility  of  the 
results  to  participants. 

Finally,  federal  agencies  could  explore  ways  of  fostering 
direct  resource  sharing  and  research  cooperation  within 
industry.  It  is  likely  that  industrially-based  research 
centers  with  users  participating  in  the  project  development 
process  from  the  outset  would  generate  highly  useful  results. 

It  is  not  apparent  that  such  consortia  gain  by  being  entirely 
university  based. 
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Developing  collaborative  programs  of  generic  research  appears  to 
be  one  of  the  best  ways  of  insuring  the  regular  transfer  of  results  to 
contexts  of  application.  Further,  such  programs  receive  substantial 
industrial  support.  The  need  for  government  involvement  in  the  process 
seems  to  be  two-fold.  First,  while  early  participation  in  an  R&D 
process  usually  gives  a  firm  a  competitive  edge  in  product  development 
and  marketing,  it  is  well  established  that  these  advantages  are  not 
sufficient  to  recover  full  costs  of  the  research  (e.g.,  Pavitt,  1979). 
Rather,  the  social  benefits  from  industrial  innovation  outrun  capturable 
rates  of  return.  (One  industrial  representative  commented  that  it  is 
ideal  to  become  the  second  company  producing  an  innovative  product.) 
Second,  institutional  mechanisms  do  not  exist  for  facilitating 
industrial  research  consortia.  Rather,  current  practices  foster 
competition  and,  as  one  industrial  engineer  put  it,  "hoarding  rather 
than  sharing"  of  results.  Thus  approaches  need  to  be  defined  for 
supporting  industries  to  collaborate  in  the  conduct  of  long-term  generic 
research  from  which  all  could  benefit,  but  which  few  could  justify 
individually. 

THE  BROADER  CONTEXT  FOR  POLICY  OPTIONS 

At  many  points,  the  preceding  discussion  of  policy  alternatives 
raises  issues  that  go  beyond  information  dissemination  procedures 
themselves  and  involve  the  broader  context  in  which  such  policies  are 
embedded.  Information  transfer  policies  cannot  be  formulated  apart  from 
attention  to  the  institutional,  regulatory,  and  economic  environment. 
Although  an  investigation  of  macroenvironmental  issues  bearing  on 
information  dissemination  policies  and  practices  would  go  well  beyond 
the  scope  of  the  present  study,  we  briefly  describe  the  most  recurrent 
concerns  our  study  uncovered. 

A  national  dissemination  policy.  Perhaps  the  most  frequently  raised 
broad  concern  is  for  a  coordinated  national  information  transfer  policy. 
The  United  States  does  not  have  one  (Bearman,  1981) .  Nor  is  there 
cabinet  level  representation  for  the  independent  science  and  technology 
agencies.  While  discussions  and  literature  reviews  yielded  no  clear 
consensus  about  specific  policy  vehicles,  we  found  considerable  support 
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for  developing  an  integrated  knowledge  transfer  system  within  the 
context  of  a  national  scientific  and  technical  information  dissemination 
policy . 

Resolution  of  roles  and  rights.  A  second  set  of  concerns  involves 
sorting  out  roles  and  rights  in  the  dissemination  process  for 
researchers,  government  agencies,  scientific  organizations,  and  private 
for-profit  firms  that  provide  "value-added"  services  in  information 
distribution. 

One  problem  is  how  the  costs  of  computerized  information  searches 
should  be  covered.  Our  historical  review  (Appendix  B)  calls  attention 
to  the  fact  that  if  government  agencies  offer  low-cost  services,  they 
are  seen  as  competing  with  private  industry;  if  they  do  not,  users  with 
limited  budget  resources  may  not  be  able  to  access  nonproprietary 
scientific  and  technical  information. 

Copyright  is  another  problem  area.  Taking  advantage  of  online 
information  resources  by  permitting  users  to  access  textual  data,  and  by 
enabling  them  to  retrieve  and  print  (or  otherwise  manipulate)  the 
portion  they  need,  raises  important  questions  about  how  rights  of 
authors  and  publishers  are  to  be  protected.  These  kinds  of  questions 
need  careful  exploration  in  relation  to  dissemination  policies  and 
practices . 

Tax  and  regulatory  policies.  Although  tax  incentives  and  regulatory 
changes  do  not  automatically  lead  to  changes  in  knowledge  transfer, 
several  recent  events  affecting  that  process  were  highlighted  in 
discussions  and  related  literature. 

For  instance,  tax  incentives  for  support  of  research  and 
development  were  part  of  the  Economic  Recovery  Act  of  1981.  While  most 
sources  believe  more  economic  incentives  for  industrial  support  of 
research  are  desirable,  the  effects  of  the  current  tax  initiative  should 
be  assessed. 

Similarly,  patent  regulations  were  revised  in  1981  to  give 
industries  that  develop  products  more  exclusive  rights  (even  though  they 
make  use  of  nonproprietary  information) ,  on  grounds  that  expansion  in 
proprietary  rights  would  encourage  inventive  activities.  Many  contend 
that  further  revision  of  patent  law  would  be  desirable.  However,  the 
effect  of  the  1981  change  has  yet  to  be  evaluated. 
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Antitrust  legislation  is  regarded  as  a  potential  barrier  to 
industrial  resource  sharing  and  research  collaboration.  In  particular, 
many  discussants  predicted  the  deregulation  of  the  communications 
industry  and  its  effect  on  Bell  Laboratories  would  have  a  strong 
negative  influence  on  industrially  based  resource  sharing  and  generic 
research.  This  question  likewise  merits  study. 

Our  discussion  of  the  broader  context  of  dissemination  policies  is 
not  meant  to  be  complete.  Rather  it  is  intended  to  provide  specific 
examples  of  macroenvironmental  issues  that  bear  on  the  success  of 
efforts  to  translate  the  results  of  federally  funded  research  and 
development  into  process  and  product  improvements. 

IMPLICATIONS  FOR  NSF 

The  discussion  of  policy  alternatives,  we  believe,  suggests  a 
number  of  directions  open  to  NSF  for  improving  the  dissemination  of 
scientific  and  technical  information.  Below  we  indicate  what  we  see  as 
the  most  promising  first  steps. 

Research  on  advanced  information  technology.  NSF's  strongest 
influence  appears  to  be  exerted  through  the  research  it  funds.  To 
enable  future  dissemination  programs  to  take  full  advantage  of  advanced 
information  technology,  NSF  could  solicit  and  support  two  sorts  of 
research.  Pilot  or  field-experimental  projects  might  address  the 
implementation  of  computer-mediated  contact  networks  (project-based, 
user-based,  and/or  systems  linking  agencies,  researchers  and  users). 
Experimental  development  projects  could  design  user-guided  querying 
tools,  knowledge  bases  to  support  intelligent  search  of  textual  or 
numeric  data,  or  interfaces  capable  of  assisting  users.  The  former 
efforts  attempt  to  extend  the  known  benefits  of  informal  interaction  in 
the  scientific  communication  process,  and  the  latter  bring  advances  in 
artificial  intelligence  and  expert  systems  research  to  bear  on  problems 
of  effective  knowledge  transfer. 

Extension  of  current  research  programs.  According  to  discussions 
conducted  for  this  study  and  to  published  results,  NSF's  support  of 
collaborative  industry/university  research  programs  shows  the  utility  of 
bringing  knowledge  producers  and  consumers  into  direct  interaction. 
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These  efforts  could  be  continued,  and  alternative  mechanisms  for 
stimulating  industrial  cooperation  in  the  conduct  and  support  of 
intermediate  research  might  be  explored.  A  second  area  of  effort  that 
might  be  expanded  is  empirical  research  on  information  transfer  and 
knowledge  utilization.  Much  of  what  is  known  about  information 
dissemination  from  an  empirical  and  comparative  standpoint  comes  from 
NSF-supported  studies.  Several  discussion  participants  thought 
behavioral  research  of  this  type  could  be  expanded. 

Project  requirements  that  improve  transferability  of  results. 

Program  solicitation  announcements  and  proposal  review  criteria  might  be 
revised  so  that  projects  will  have  a  greater  likelihood  of  being  funded 
if  they  include  procedures  for  interaction  with  potential  users  of  the 
resulting  research.  Above  we  have  suggested  several  ways  of  achieving 
user  participation:  including  an  advisory  committee  of  potential  users 
who  provide  feedback  and  review  at  regular  intervals  throughout  a 
project;  including  information  broker  activities  for  researchers  at  the 
conclusion  of  the  work;  and  identifying  information  entrepreneurs  within 
relevant  user  settings  to  whom  results  could  be  directly  disseminated. 
These  options  are  not  exclusive;  however,  some  will  be  more  appropriate 
than  others  for  any  given  project.  Although  any  of  these  activities 
will  increase  project  budgets,  the  utilization  payoff  may  also  be 
considerable . 

Improvements  in  collected  information.  Information  supply-side 
options  could  be  reviewed  to  determine  which,  if  any,  could  be 
implemented  at  small  cost  to  improve  the  usefulness  of  extant  scientific 
and  technical  information.  For  example,  the  NTIS  database  might  provide 
an  appropriate  foundation  for  low-cost  experimentation  on  co-citation- 
based  report  grouping  as  a  way  of  helping  users  conduct  more  selective 
information  searches.  Or,  NSF  could  undertake  to  improve  future  NTIS 
abstracts  by  including  more  information  about  research  methodology  and 
indicators  of  data  availability.  Further,  the  NTIS  retrieval  system 
might  be  used  as  a  testbed  for  exploring  the  union  of  data  retrieval 
with  data  manipulation  algorithms.  In  any  case,  discussants  suggested 
that  internal  NSF  activities  of  this  sort  might  become  models  and  guides 
to  other  agencies . 
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Critical  evaluations  of  numeric  data  deserve  special  attention 
because  they  are  very  costly  to  create  and  because,  once  established, 
evaluated  data  sources  require  regular  updating  and  maintenance. 

Further,  they  could  become  de  facto  subsidies  to  large  corporations, 
their  primary  users.  On  the  other  hand,  they  may  have  extremely  high 
utility.  Thus  data  evaluation  could  be  undertaken  where  user  groups  can 
be  identified  who  have  major  unmet  needs  for  specific  data  sources 
appropriately  filled  by  government  activities  (in  some  instances  costs 
may  be  at  least  partially  recoverable  through  user  charges) .  Evaluated 
data  sources  integrate  information  from  different  databases  supported  by 
different  agencies.  While  discussants  agreed  that  NSF  should  encourage 
the  strengthening  of  data  evaluation,  they  were  less  clear  about  how  to 
establish  priorities  and  about  appropriate  NSF  roles  in  these 
activities.  Many  believe  that  Al-based  technology,  yet  to  be  developed, 
holds  most  promise  for  solving  data  evaluation  problems. 

Information  system  standards.  Issues  of  standardization  for 
computer  and  communications  systems  need  to  be  addressed  by  an  inter¬ 
agency  body  of  some  sort.  While  premature  moves  toward  standardization 
can  depress  innovation  and  unduly  constrain  options,  waiting  too  long 
can  make  the  benefits  of  cost  reductions  in  electronic  technology  more 
and  more  difficult  to  realize  as  printed  matter  proliferates.  Because 
the  development  of  viable  standards  is  painstaking  and  will  require  a 
long  time,  it  is  generally  agreed  that  such  efforts  should  begin  now. 

NSF  might  take  the  lead  in  initiating  them,  in  order  to  bring  potential 
technology- intensive  options  for  information  dissemination  closer  to 
reality. 

Information  policy  leadership.  There  is  widespread  recognition 
that  the  United  States  needs  coordinated  national  science  information 
policies,  of  which  dissemination  policies  are  a  part.  However, 
discussions  and  literature  reviews  led  to  no  determinate  conclusions  as 
to  NSF's  role  in  this  area.  Several  believe  NSF  should  serve  as  a 
national  leader  in  information  policymaking,  acting  not  only  to  promote 
basic  research  but  also  to  insure  that  excellence  in  science  is 
adequately  translated  into  product  and  process  innovation.  Others  are 
skeptical  about  whether  NSF  could  carry  out  such  a  broadened  leadership 
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role.  They  do  not  want  to  see  a  weakening  of  NSF's  support  for  basic 
research,  and  are  unsure  of  whether  this  responsibility  is  consistent 
with  the  development  of  aggressive  knowledge  transfer  policies.  The 
question  of  how  best  to  establish  a  high  level  base  for  scientific  and 
technical  knowledge  transfer  policy  should  be  investigated  and  resolved. 
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APPENDIX  A. 

DIRECTIONS  FOR  FUTURE  DISSEMINATION  POLICY  RESEARCH 


In  this  report  we  have  hardly  scratched  the  surface  of  a  large  and 
enormously  important  area.  Obviously,  a  good  deal  of  study  remains  to 
be  done  to  provide  a  detailed  basis  for  improved  federal  information 
transfer  policies.  In  this  appendix,  we  discuss  a  framework  for  future 
research  by  drawing  on  the  issues,  problems,  and  options  we  have 
identified  and  discussed  above. 

We  believe  that  continued  study  of  information  dissemination  policy 
could  productively  pursue  the  following  four  topics,  which  we  discuss  in 
turn : 

•  Formal  evaluation  of  current  programs  and  options 

•  Technology  assessment 

•  The  regulatory- legal  environment 

•  Comparative  national  policies  and  transborder  information  flow 

FORMAL  EVALUATION:  ILLUSTRATIVE  CRITERIA 

Section  II  above  informally  evaluates  major  current  policies  and 
options  for  the  future.  A  more  structured  and  rigorous  approach  is 
needed  in  subsequent  work.  Especially  important  is  analysis  of  costs 
and  benefits  as  well  as  roles  for  the  federal  government  vis-a-vis  the 
private  sector.  A  key  problem  is  to  specify  the  circumstances  under 
which  the  government  should  pursue  particular  information  transfer 
programs,  rather  than  leaving  these  activities  to  the  private  market 
place.  The  proliferation  of  information  technologies,  systems  and 
services  has  greatly  complicated  this  question. 

Addressing  these  and  other  considerations  requires,  as  the  first 
step,  a  well-defined  set  of  evaluation  criteria.  We  offer  here  a 
preliminary  set  of  criteria  to  assist  in  this  task  (see  Table  1.) 
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Table  1 

SUMMARY  OF  CRITERIA  FOR  EVALUATING  DISSEMINATION  POLICIES 

AND  PRACTICES 


GENERAL  CRITERIA 


Cost,  Benefits,  and  Equity 
Institutional  Feasibility 
Technical  Feasibility 


SPECIFIC  CRITERIA 


ABILITY  TO  MAKE 
INFORMATION  AVAILABLE 

Comprehensiveness 
Assurance  of  Access 
Breadth  of  Access 
Timeliness  of  Access 
Continuity 


EVALUATIVE 

ABILITY 

Assurance  of 

-  Reliability 

-  Validity 

-  Documentation 
Integrative  Ability 
Discriminative  Ability 


RESPONSIVENESS  TO 
USER  ENVIRONMENT 

Multidisciplinary 

Retrieval  Capability 
Problem -Relevance 
Amenability  to 
User  Direction 
Facilitation  of 
Interaction 


General 

Three  criteria  are  so  general  that  they  enter  all  discussions  of 
knowledge  transfer  policies  and  practices.  They  represent  conditions 
that  present  and  future  programs  should  meet.  At  the  same  time,  they 
raise  special  issues  related  to  the  use  of  advanced  information 
technology  to  support  Federal  dissemination  efforts. 

Cost  and  equity  are  obviously  important  criteria  for  evaluating 
quality  assurance  and  information  transfer  processes.  Unfortunately, 
benefits  are  harder  to  measure  than  costs.  The  latter  usually  can  be 
estimated  tolerably  well  by  adding  up  hardware  and  software  costs, 
employee  and  other  staff  costs,  operating  expenses,  and  the  like.  But 
the  benefits  to  users  of  scientific  and  technical  information  generated 
by  government  funding  of  basic  research  are  more  elusive.  In  principle, 
a  good  measure  is  the  amount  the  users  would  be  willing  to  pay,  but 
determination  of  this  amount  itself  poses  problems.  Moreover,  it  is 
hard  to  identify  all  possible  beneficiaries.  Some  users  may  get  their 
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information  from  other  users,  and  identifying  secondary  users  and 
estimating  the  value  of  the  information  to  them  as  well  pose  additional 
complications . 

One  way  to  choose  the  most  cost-effective  options  might  be  to 
compare  the  costs  of  competing  processes  whose  benefits  appear  to  be 
roughly  the  same  and  select  the  less  expensive  ones.  In  other  cases, 
large  differences  in  costs  or  benefits  may  make  decisions  easier. 
Generally,  federal  policies  presuppose  the  social  value  of  open 
scientific  and  technical  literature  and  do  not  cost- justify  it. 

Equity,  too,  has  been  an  important  consideration.  For  example, 
federal  programs  assisting  the  private  sector  generally  give  special 
priority  to  small  business  and  minority-owned  enterprises.  These  groups 
may  face  special  difficulties  in  fully  utilizing  scientific  and 
technical  information.  The  cost  burden  of  hardware,  software,  and  staff 
training  may  be  high  for  small  businesses  that  cannot  exploit  the  scale 
economies  available  to  larger  entities.  Federal  policy  may  seek 
especially  to  assure  that  transfer  processes  be  designed  to  meet  the 
needs  of  small  nonacademic  organizations,  since  they  seem  to  be  least 
adequately  served  by  traditional  dissemination  mechanisms.  This 
emphasis  could  be  embodied,  for  instance,  in  transfer  processes  that 
require  only  small  up-front  investments  by  users  or  modest  staff  skill 
requirements.  On  the  other  hand,  for  larger  user  organizations,  such 
processes  may  be  less  effective  than  those  that  do  require  larger  budget 
and  manpower  resources.  A  major  policy  issue,  then,  is  how  to  enact 
effective  and  flexible  options  that  will  serve  a  range  of  user 
organizations . 

Institutional  Feasibility.  Evaluations  of  current  programs  and  future 
options  must  take  into  account  the  institutional  difficulties  of 
supporting  or  implementing  them.  These  difficulties  relate  to 
appropriate  roles  for  the  federal  government  and  the  private  sector,  and 
the  internal  operation  of  the  federal  government. 

The  fundamental  rationale  for  government  involvement  in 
dissemination  is  the  "public  goods"  character  of  information.  A  "pure" 
public  good  is  one  whose  use  by  one  individual  does  not  reduce  the 
supply  available  to  others.  Once  produced,  it  is  available  to  everyone 
at  no  additional  cost.  (The  clearest  example  is  national  defense.)  The 


50 


private  marketplace  generally  will  produce  less  than  the  socially 
appropriate  or  "optimal"  amount  of  a  public  good.  Roughly  speaking,  the 
optimal  level  of  information  is  that  for  which  the  value  to  the  user  of 
the  last  unit  provided  is  equal  to  the  additional  cost  of  providing  it. 
However,  a  uniform  price  to  all  users  that  is  equal  to  the  additional 
cost  of  serving  each  would  not  generate  sufficient  revenue  to  cover  the 
producer's  total  cost.  The  producer  could  charge  a  price  in  excess  of 
the  additional  cost  to  cover  total  cost,  but  fewer  units  of  information 
would  be  sold  at  the  higher  price.  Also,  some  users  would  be  deprived 
of  the  information,  even  at  a  fair  price  (i.e.,  one  that  only  supports 
cost  recovery) . 

Information  of  value  to  a  wide  variety  of  users  has  a  stronger 
public-goods  character- -and  hence  a  stronger  rationale  for  federal 
support- -than  does  information  tailored  to  the  needs  of  a  few.  Whether 
or  not  information  of  private  utility  is  produced  depends  on  whether  the 
value  to  any  individual  user  at  least  equals  the  additional  cost  of 
producing  it.  In  these  situations,  the  private  marketplace  is  generally 
relied  upon  to  produce  (or  not  to  produce)  the  information.  Thus,  in 
examining  the  relationship  of  dissemination  policy  options,  federal  and 
private-sector  roles,  a  major  criterion  is  the  extent  of  their  public- 
goods  characteristics.1 

If  there  is  a  legitimate  federal  role  in  particular  information 
transfer  options,  it  is  important  also  to  assess  feasibility  in  terms  of 
the  internal  operation  of  the  government.  Options  that  can  easily  be 
implemented  by  individual  agencies  are  more  institutionally  feasible 
than  those  that  would  require  close  coordination  among  agencies,  new 
statutory  authority,  or  creation  of  a  wholly  new  agency.  Similarly,  the 
successes  and  failures  of  existing  programs  depend  in  part  on  the  ease 
with  which  they  fit  into  existing  agency  interests  and  mandates. 

Technical  feasibility.  Any  dissemination  policy  should  be  capable 
of  implementation  with  currently  available  technology  on  an  efficient 

1  This  distinction  between  types  of  information  is  similar  to  that 
between  "basic"  and  "applied"  research,  where  a  clearer  case  for  federal 
support  of  basic  research  can  be  made  on  grounds  of  its  stronger  public- 
goods  characteristics.  Of  course,  the  analogy  is  weakened  by  the 
blurring  of  the  basic-applied  distinction  and  the  need  for  "generic" 
research. 
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and  effective  basis.  Normally  this  implies  gradual  or  incremental 
evolutionary  policy.  However,  we  believe  that  government  investment  in 
information  technology  has  emphasized  capture  and  storage  at  the  expense 
of  more  efficient  access  techniques  (particularly  in  the  realm  of 
dissemination  to  nongovernment  users  of  the  data) .  Government 
investment  in  technology  appropriate  for  information  dissemination  has 
been  substantial.  For  example,  the  DARPA  Network  has  been  an  ongoing 
government  technology  investment  for  over  a  decade.  The  results  in  the 
realm  of  information  dissemination  have  been  nothing  short  of 
spectacular.  An  entire  research  community  (computer  science)  uses  this 
network  primarily  for  information  transfer.  This  example  suggests  that 
much  government  technology  investment  has  potential  for  broader 
realization  of  more  efficient  dissemination  vehicles.  Before  any  new 
inventions  are  applied  to  information  transfer,  an  inventory  of  what 
already  exists  and  the  generation  of  appropriate  plans  for  adaptation 
and  broader  implementation  are  in  order. 

Ability  To  Make  Information  Available 

As  we  have  indicated,  existing  policies  and  practices  have  been 
relatively  successful  in  making  scientific  and  technical  information 
publicly  available.  In  fact,  observers  of  the  information  explosion 
have  frequently  commented  that  present  utilization  problems  stem  from 
having  too  much  information  available,  rather  than  too  little.  However, 
because  increasing  use  of  electronic  information  systems  in 
dissemination  procedures  unquestionably  affects  user  access,  policy 
options  could  be  assessed  in  terms  of  the  following  availability-related 
evaluation  criteria: 

•  Comprehensiveness.  Federal  dissemination  policies  should 
yield  comprehensive  coverage  of  advances  in  research  and 
development  within  every  discipline;  no  information  should  be 
lost . 

•  Assurance  of  access.  The  assurance  of  access  may  seem  so 
inherent  in  the  concept  of  information  transfer  as  to  be  a 
trivial  criterion.  However,  there  is  growing  evidence  that 
users  experience  considerable  difficulty  in  actually  locating 
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and  acquiring  material  that  is  in  principle  "available." 
Information  structures,  search  tools  and  dissemination  media 
should  promote  confident  retrieval. 

•  Breadth  of  access.  Because  potential  consumers  of  scientific 
and  technical  information  are  diverse,  dissemination  efforts 
should  reach  multiple  user  populations. 

•  Timeliness.  Dissemination  policies  should  make  new  scientific 
and  technical  information  available  as  it  becomes  known,  or  it 
will  be  of  less  value  for  stimulating  performance  and 
productivity  improvements. 

•  Continuity.  While  the  public-  and  private-sector  organizations 
and  services  involved  in  knowledge  transfer  activities  may 
change  considerably,  federal  dissemination  policies  should 
provide  for  long-term  continued  availability  of  bodies  of 
scientific  and  technical  information. 

Evaluative  Ability 

The  issue  of  information  quality  assurance  has  attracted 
considerable  attention  in  recent  years.  Providing  quality  controls  and 
guidelines  has  been  seen  as  a  way  of  increasing  users'  ability  to  cope 
with  the  vast  and  growing  array  of  scientific  and  technical  information. 
The  policy  challenge  is  two-fold:  to  promote  quality  in  primary 
information  products  (with  special  attention  to  the  value  of 
physicochemical  reference  data) ;  and  to  provide  users  with  more 
selective  search  tools,  so  that  their  ability  to  impose  quality  filters 
on  available  information  (whether  numeric  or  textual)  is  improved.  The 
criteria  described  below,  which  place  a  premium  on  evaluating  scientific 
and  technical  information,  are  suggested  as  ways  of  appraising  the 
ability  of  policy  alternatives  to  meet  these  aims. 

Assurance  of  reliability,  validity,  and  documentation.  Information 
dissemination  policies  should  encourage  the  adherence  of  all  primary 
scientific  and  technical  information  products  to  the  following 
objectives,  which  seem  to  define  the  quality  dimension: 
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•  Reliability,  or  the  extent  to  which  identical  operations  if 
repeated  would  produce  identical  data. 

•  Validity ,  or  the  viability  of  the  logic  underlying  inferences 
and  conclusions  drawn  from  the  data. 

•  Documentation,  or  the  provision  of  clear  descriptions  of 
research  design,  variables,  instruments,  and  procedures,  along 
with  explanations  as  to  how  results  are  identified,  recorded 
and  organized.  As  we  have  noted,  standards  for  evaluation  of 
reliability  and  validity  differ  in  different  research 
disciplines  (e.g.,  physical  properties  vs.  social  data). 
However,  full  methodological  documentation  is  a  uniform 
requirement  that  will  enable  users  to  make  better  evaluations 
of  research  results  in  all  domains. 

Integrative  ability.  Federal  dissemination  policies  should  promote 
transfer  procedures  and  services  that  can  integrate  or  at  least  collate 
related  studies,  thus  enabling  the  recognition  of  duplication,  the 
comparison  of  results  and  conclusions  from  different  studies,  and  the 
reconciliation  of  inconsistencies. 

Discriminative  ability.  Federal  dissemination  policies  should 
facilitate  selective  search  and  retrieval  of  research  output  within  a 
given  line  of  inquiry.  While  information  access  should  not  be 
constrained,  users  should  be  provided  with  some  mechanism  for 
determining  the  most  relevant  and  important  contributions  to  knowledge. 

Responsiveness  to  User  Environment 

Federal  information  transfer  policies  could  be  assessed  in  terms  of 
their  ability  to  promote  the  aggressive  matching  of  information  delivery 
and  information  needs.  This  "user-responsiveness"  dimension  of 
information  transfer  (unlike  the  quality  control  dimension)  does  not 
have  a  long  history  of  discussion  in  scientific  literature. 

Nevertheless,  recent  empirical  and  analytic  studies  suggest  several 
criteria  related  to  user-orientation  that  may  enhance  information 
utility . 
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Multidisciplinary  retrieval  capability  is  the  most  frequently  cited 
criterion  in  discussions  of  the  usability  of  scientific  and  technical 
information.  While  knowledge  needs  that  cross  disciplinary  boundaries 
have  sometimes  been  thought  to  characterize  only  applied  use,  many 
current  advances  in  basic  research  and  development  (e.g.,  in 
biotechnology,  computer  science)  depend  on  interdisciplinary  knowledge 
as  well.  This  criterion  does  not  imply  that  knowledge  producers  should 
not  report  results  in  the  language  of  their  research  disciplines; 
however,  information  system  structures  and/or  access  mechanisms  (e.g., 
keywords,  indexes)  should  provide  multiple  routes  to  primary  products. 

Problem-relevance.  To  promote  usability  of  scientific  and 
technical  information,  dissemination  policies  also  need  to  promote 
problem-focused  retrieval.  The  great  difficulty  posed  by  this  criterion 
is  that  the  language,  methods,  assumptions,  and  values  of  the  problem 
domain  are  likely  to  be  quite  different  from  those  of  relevant  research 
and  development  domains.  In  addition,  information  structures  and  search 
tools  are  not  designed  to  help  users  filter  primary  products  for 
instrumental  relevance. 

Amenability  to  user  direction.  There  is  an  even  more  basic  concern 
to  be  faced  in  the  effort  to  transfer  problem-relevant  information:  Only 
users  are  well  equipped  to  determine  what  is  relevant  to  their  problems. 
Moreover,  prejudgments  of  relevance  and  other  efforts  to  screen 
information  run  the  risk  of  curtailing  innovative  or  serendipitous 
information  use.  On  the  other  hand,  user  involvement  in  the  knowledge 
transfer  process  has  consistently  been  associated  with  greater 
utilization.  Therefore,  federal  dissemination  policies  should  increase 
flexibility  and  advance  user  participation  in  information  search  and 
retrieval  activities  so  that  these  processes  become  more  user-guided. 

Facilitation  of  interaction.  Finally,  one  result  that  emerges 
strongly  in  research  on  knowledge  transfer,  regardless  of  domain,  is 
that  formal  one-way  channels  are  not,  in  themselves,  sufficient  to 
promote  utilization.  Rather,  communicating  information  is  much  more 
effective  when  it  occurs  as  an  exchange,  allowing  for  scientific  dialog, 
iterative  feedback,  and  refinement  of  implications.  Consequently, 
federal  dissemination  policies  should  encourage  the  development  of 
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interactive  information  environments  that  enable  contact  among  users  and 
between  users  and  producers  of  scientific  and  technical  knowledge. 

TECHNOLOGY  ASSESSMENT 

We  have  concluded  that  much  improvement  in  information  transfer  and 
quality  assurance  is  possible  with  existing  technology.  Nevertheless, 
it  is  important  to  explore  the  potential  impact  of  new  technologies  in  a 
number  of  ways.  First,  they  may  provide  greater  flexibility  in  meeting 
specific  needs.  Today  heavy  reliance  is  placed  on  the  relatively  rigid 
subject-matter,  key-word  approach  to  search  large  data  bases--an 
approach  whose  shortcomings  we  have  emphasized.  Expert  systems,  high 
level  languages,  and  other  outcomes  from  artificial  intelligence 
research  may  make  other  approaches  economically  and  technically 
feasible.  For  example,  they  might  give  rise  to  the  design  of  more 
heuristic  approaches,  where  users'  iterative  inquiries  trigger  more 
sophisticated,  goal-oriented  searches.  Or,  message  networks  may  allow 
the  development  of  self -configured  groups  that  readily  share  and  build 
on  one  another's  information  resources. 

Second,  the  opportunities  and  problems  posed  by  use  of  new 
technologies  may  both  reduce  and  intensify  institutional  barriers.  The 
degree  to  which  federal  agencies  would  have  to  cooperate  to  make  best 
use  of  new  technologies,  the  merits  of  a  single  agency  having  primary 
responsibility  for  interagency  coordination  of  quality  assurance  and 
transfer  policies,  and  the  viability  of  today's  decentralized  system, 
could  all  be  affected.  Moreover,  appropriate  roles  for  the  public  and 
private  sectors  may  change.  We  noted  in  our  discussion  of  evaluation 
criteria  that  the  more  tailored  is  information  to  meet  the  needs  of  the 
individual  user,  the  stronger  is  the  rationale  for  its  provision  by  the 
private  sector.  To  the  extent  that  new  technology  makes  more  feasible 
the  tailoring  of  information  to  meet  individual  needs,  the  case  is 
strengthened  for  having  these  new  functions  performed  by  the  private 
sector.  On  the  other  hand,  advanced  information  technology  may  permit 
users  to  tailor  it  themselves.  Finally,  the  dependence  of  greater  use 
of  information  technology  for  dissemination  processes  on  the  development 
of  standards  needs  careful  appraisal. 
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REGULATORY  AND  LEGAL  ISSUES 
Legal  Liability 

The  issue  of  legal  liability  is  particularly  relevant  to  the  task 
of  quality  assurance,  whose  importance  we  have  emphasized  throughout 
this  report.  Under  what  conditions  should  an  organization  that 
evaluates  information  for  others  be  held  legally  liable  if  damages  occur 
because  the  evaluated  information  is  misleading,  misused,  or  faulty? 
Westbrook  and  Rumble  (1983)  note  that  about  25  percent  of  lawsuits  on 
new  products  allege  engineering  negligence.  Because  some  of  these  cases 
arise  from  data  problems,  these  authors  urge  that  the  issue  of 
information  liability  be  assessed;  we  agree.  In  particular,  further 
study  ought  to  be  given  to  defining  appropriate  limits  of  liability  and 
exploring  the  extent  to  which  legal  liability  discourages  organizations 
(for  example,  professional  societies)  from  taking  a  more  active  role  in 
quality  assurance. 

Copyright 

Copyright  issues  have  posed  difficulties  for  many  years  with 
respect  to  protection  of  data  bases  and  software,  among  other  things. 

The  National  Academy  of  Engineering  (1981)  complained  that,  under 
existing  1976  copyright  law,  software  developers 

have  had  to  protect  their  investment  by  restricting  its  use 
through  nondisclosure  agreements  or  withholding  it  completely. 
Similarly,  data  base  developers  and  services  are  reluctant  to 
invest  heavily  in  offering  new  information  services  in  view  of 
present  ambiguities  in  the  law. 

The  1980  copyright  law  amendments  adopted  the  recommendations  of 
the  National  Commission  on  New  Technological  Uses  of  Copyrighted  Works 
that  "all  computer  programs,  fixed  in  any  method  and  performing  any 
function,  be  included  within  copyright  protection."  Nevertheless,  the 
problems  of  the  sort  expressed  by  the  National  Academy  of  Engineering 
persist  because  of  difficulties  of  enforcement,  ambiguities  in  the  law, 
and  new  legal  issues  that  arise  because  of  continuing  technological 
advances.  As  one  example,  the  recent  Betamax  decision  may  expand  the 
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legal  interpretation  of  "fair  use"  for  noncommercial  copying,  but  the 
new  boundaries  of  fair  use  will  surely  be  the  subject  of  continual 
litigation . 

We  have  identified  two  areas  in  which  further  study  is  especially 
needed : 

•  Empirical  examination  of  the  degree  to  which  both  legal  and 
illegal  copying  occurs.  Beyond  scattered  anecdotal  evidence, 
surprisingly  little  is  known  about  the  extent  to  which 
intellectual  property  is  being  copied  and  used.  Thus,  the  true 
magnitude  of  the  copyright  problem  itself  is  in  doubt. 
Systematic  empirical  analysis  is  both  feasible  and  would 
greatly  assist  the  formulation  of  well-conceived  future 
copyright  policy. 

•  Analysis  of  new  legislative  initiatives  that  might  be 
undertaken,  based  on  economic  efficiency  and  other  criteria. 
Careful  attention  should  be  given  to  the  extent  to  which 
development  and  use  of  intellectual  property  is  less  than 
socially  optimal  because  of  problems  of  defining  and  protecting 
property  rights ,  and  what  can  be  done  about  it .  Such 
considerations  are  especially  important  in  view  of  the 
potential  for  retrieving  and  using  parts  of  online  primary  or 
secondary  research  products . 

Antitrust  Considerations 

Although  the  Justice  Department  has  stated  that  it  has  never 
challenged  a  pure  research  venture  without  corollary  restraints  in 
trade,  cooperation  among  private  firms  in  R&D  activities  has  been 
hampered  in  the  past  by  fears  of  possible  antitrust  violations. 

However,  joint  R&D  ventures  are  becoming  increasingly  popular,  because 
of  congressional  and  administration  encouragement  and  proposals  by  the 
Department  of  Justice  for  antitrust  reforms.2  The  growth  of  joint 
ventures  has  implications  for  both  the  production  and  utilization  of 

2  An  excellent  survey  of  recent  developments  is  contained  in  the 
National  Journal,  May  14,  1983,  pp.  1000-1005. 
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information  from  federally  funded  research.  Especially,  these 
cooperative  arrangements  may  increase  the  demand  for  information  and 
thus  put  additional  pressure  on  federal  agencies  to  improve  their 
information  utilization  activities.  Moreover,  it  is  uncertain  to  what 
degree  firms  will  be  able  to  share  information  in  both  their  basic  and 
applied  research  activities  without  violating  antitrust  laws. 

Studies  should  be  undertaken  as  to  the  degree  to  which  (a) 
antitrust  barriers  exist  to  the  sharing  and  use  of  information  that 
would  be  in  the  public  interest,  (b)  how  these  barriers  might  best  be 
overcome,  (c)  how  the  growth  of  private  joint  ventures  will  affect  the 
demand  for  information  from  federal  agencies,  and  (d)  how  the 
information  needs  of  these  ventures  can  best  be  accommodated,  either  by 
federal  or  private  sector  initiatives. 

Tax  Considerations 

Past  studies  of  information  policies  have  noted  two  basic  ways  to 
improve  the  utilization  of  information:  (a)  strengthening  the  supply 
side  by  encouraging  improved  data  collection,  evaluation,  and  user 
access,  and  (b)  strengthening  the  demand  side. 

One  way  to  strengthen  the  demand  side  is  to  provide  tax  credits  or 
other  tax  advantages.  Until  1981,  federal  law  did  not  provide  a  tax 
credit  specifically  for  research  or  experimental  expenditures.  However, 
the  Economic  Recovery  Act  of  1981  provides  that  a  taxpayer's  investment 
in  machinery  and  equipment  employed  in  research  or  other  experimental 
activities  is  eligible  for  the  investment  tax  credit  to  the  same  extent 
as  investment  in  other  business  activities.  In  addition,  monetary 
support  granted  to  research  institutions  producing  nonproprietary 
information  likewise  receives  a  tax  credit. 

How  effective  tax  credits  will  be  in  stimulating  user  demand  for 
information  remains  to  be  seen.  Study  is  needed  of  the  likely  effect  of 
tax  credits  and  other  tax  provisions  on  the  demand  for  information 
(whether  supplied  by  federal  or  other  sources) .  Such  tax  policies  might 
have  either  a  trivial  or  significant  effect,  in  comparison  with  changing 
domestic  and  foreign  market  conditions,  for  the  output  of  private-sector 
R&.D  activities--and  hence  on  the  demand  for  information  from  federally 
funded  and  other  sources . 
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Patent  Policy 

Amendments  to  the  patent  and  trademark  laws  in  1981  replaced  a 
melange  of  26  different  agency  policies  on  vesting  of  patent  rights  in 
government -funded  research  with  "a  single,  uniform  national  policy 
designed  to  cut  down  on  bureaucracy  and  encourage  private  industry  to 
utilize  government  funded  inventions  through  the  commitment  of  the  risk 
capital  necessary  to  develop  such  inventions  to  the  point  of  commercial 
application"  (U.S.  Code,  Congressional  and  Administrative  News,  Vol.  5, 
1980,  p.  6462).  Ownership  of  patent  rights  in  government -funded 
research  by  nonprofit  or  small-business  contractors  now  vests  in  these 
organizations.  Other  contractors  are  given  exclusive  licenses,  under 
conditions  specified  in  the  amendments,  giving  them  proprietary  rights 
previously  unavailable.  These  incentives  are  expected  to  improve  the 
incentive  of  private  sector  firms  to  acquire  and  apply  the  results  of 
federally  funded  R&D. 

Like  the  issues  of  copyright  discussed  above,  issues  arise  about 
the  extent  to  which  proprietary  rights  to  information  improve  or  degrade 
knowledge  transfer  from  federally  funded  research  activities.  On  the 
one  hand,  the  scope  of  publicly  available  information  bases  (and  perhaps 
their  usefulness)  is  reduced  because  they  do  not  include  proprietary 
material.  On  the  other  hand,  the  expansion  of  exclusive  rights  may 
encourage  the  expansion  of  inventive  activity,  and  thus  the  utility  of 
information  that,  though  developed  through  the  private  marketplace, 
results  in  products  that  serve  the  public  interest.  Study  is  needed  of 
the  implications  of  these  patent  amendments  for  information  policy  and 
of  other  changes  that  may  be  needed  in  patent  laws. 

COMPARATIVE  NATIONAL  POLICIES  AND  TRANSBORDER  INFORMATION 

Examination  of  information  policies  in  other  industrialized 
countries  could  provide  valuable  insight  into  the  opportunities  for 
improvements  in  the  United  States.  Quality  assurance  mechanisms  used 
elsewhere  might  provide  guidance  for  recognizing  opportunities  and 
avoiding  problems  in  the  quest  for  better-evaluated  information. 
Similarly,  valuable  insight  might  be  derived  from  examinations  of  ways 
in  which  foreign  organizations  interpret  and  exploit  the  utility  of  the 
open  scientific  and  technical  literature. 
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Two  frequently  voiced  concerns  are  that  (a)  foreign  firms  make 
better  use  of  information  from  U.S.  research  activities  than  do  domestic 
firms;  and  (b)  information  exchange  is  largely  a  one-way  street--foreign 
firms  gain  more  from  American-generated  information  than  do  American 
firms  from  foreign  sources.  These  concerns  are  based  on  comparisons  of 
patent  application  data  in  Japan  and  the  U.S.  and  on  the  erosion  of  the 
competitive  edge  of  domestic  firms  in  high-technology  industries. 

Foreign  countries  gain  more  from  the  United  States  than  we  do  from 
them  if  for  no  other  reason  than  that  more  basic  research  is  done  in  the 
United  States  than  in  any  other  single  country.  However,  language 
problems  also  create  an  asymmetry  in  information  use;  for  instance,  more 
Japanese  engineers  and  scientists  have  a  working  knowledge  of  English 
than  their  American  counterparts  have  of  the  Japanese  language.  In 
addition,  management  structures  and  processes  within  Japanese 
organizations  are  believed  to  promote  industrial  application  of  R&D 
innovations  while  interorganizational  collaboration  is  thought  to 
promote  information  resource  sharing.  But  whether  foreign  firms  in  fact 
are  better  able  to  utilize  information  than  American  corporations  is 
less  clear.  The  advantages  enjoyed  by  foreign  firms  may  stem  from  a 
number  of  factors,  such  as  low-cost  labor  and,  more  recently,  a  very 
strong  dollar  exchange  rate.  Study  is  needed  of  the  extent  to  which 
advantages  also  accrue  from  better  usage  of  information,  including 
mechanisms  for  using  knowledge  transfer  to  encourage  industrial 
innovation.  International  study  might  provide  valuable  insights  for  the 
generation  of  viable  policy  alternatives  for  the  knowledge  transfer 
process . 
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APPENDIX  B. 

A  HISTORY  OF  FEDERAL  INFORMATION  POLICIES1 


The  federal  government  to  date  has  not  seen  a  need  to  develop  a 
coordinated  agency-wide  system  for  disseminating  scientific  and 
technical  information.  Instead,  federal  involvement  in  this  area  has 
been  characterized  by  disparate  policy  actions  that  generally  reflect 
changes  in  society's  attitudes  about  science,  technology,  or 
information.  These  actions  have  often  been  initiated  by  executive 
branch  agencies;  sometimes  they  have  been  confirmed  by  broader 
administrative  and  legislative  mandates.  They  have  always  been  tied  to 
budgetary  considerations. 

The  history  of  these  policies  reveals  two  basic  characteristics: 
First,  although  opinions  regarding  the  optimal  level  of  federal 
commitment  have  varied,  it  has  become  axiomatic  that  scientific  and 
technical  information  is  sufficiently  valuable  to  justify  some  federal 
role  in  its  production  and  transfer.  Second,  the  primary  justification 
for  federal  support  has  rested  in  the  potential  for  converting 
scientific  and  technical  information  into  improvements  in  the  nation's 
welfare.  For  this  reason,  priorities  for  disseminating  scientific  and 
technological  information  have  usually  mirrored  national  priorities  (new 
weapons  systems,  space,  exploration,  better  crops)  and  reflected 
estimates  of  who  uses  the  information  best  (scientists  in  research 
settings,  physicians  in  hospitals,  engineers  in  industry). 

The  dimensions  of  information  transfer--availability,  quality 
assurance,  and  responsiveness  to  the  user  environment --have  had  varying 
degrees  of  federal  attention.  In  response  to  the  post-1945  information 
explosion,  federal  dissemination  policies  concentrated  on  establishing 
and  supporting  tools  to  promote  the  availability  of  information. 
Essential  as  these  efforts  have  been  in  preventing  the  loss  of  valuable 
research  findings,  they  have  not  proved  effective  as  knowledge  transfer 
needed  mechanism. 

1This  summary  draws  heavily  on  the  following  sources:  Burton  W. 
Atkinson  (1978);  Rossmassler  and  Watson  (1980);  and  NATO  (1983). 
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There  has  been  a  longstanding  awareness  that  evaluation  must  be  an 
integral  part  of  packaging  information  for  users'  needs.  The  Weinberg 
report  (1969)  underscores  the  scientific  importance  of  compacting, 
reviewing,  and  interpreting  the  literature,  both  for  basic  research 
progress  and  for  more  specialized  application.  The  evaluation  issue  is 
emphasized  in  relation  to  technical  reports  because,  although  they  are 
significant  output  vehicles  for  federal  research,  they  lack  an 
established  peer  review  process.  Consequent  such  products  are  often  not 
perceived  by  the  scientific  and  technical  community  as  prestigious 
sources  of  information.  The  challenge  of  creating  high-quality,  high- 
utility  information  by  means  of  data  evaluation  efforts  has  been  taken 
up  by  modestly  funded  federal  programs,  as  well  as  by  international 
coordinating  agencies,  professional  societies,  and  commercial 
publishers - -with  limited  success  as  yet  (Carter,  1980;  Westbrook  in 
AGARD,  1983a). 

In  the  last  15  years,  many  programs  for  disseminating  scientific 
and  technological  information  have  begun  to  address  the  difficulties 
involved  in  transferring  knowledge  to  target  audiences.  Studies  have 
recommended  analyzing  communication  among  scientists  and  engineers  to 
integrate  formal  information  channels  with  informal  communication 
patterns  (SATCOM,  1969).  Administrators  of  scientific  research  have 
come  to  understand  that  transferring  knowledge  effectively  to  users 
requires  tailoring  technical  information  to  meet  the  users'  needs. 

In  the  remainder  of  this  section,  we  review  representative  policy 
accomplishments,  failures,  and  opportunities  arising  from  federal 
activities  relating  to  information  availability,  quality  assurance,  and 
responsiveness  to  the  user  environment.  Milestones  in  the  history  of 
information  policy  are  summarized  in  Appendix  C. 

AVAILABILITY 

The  federal  government  provides  the  nation's  major  funder  of  R&D. 
In  1982,  total  U.S.  R&D  expenditures  were  estimated  at  $77.3  billion, 
and  federal  expenditures  at  $38.7  billion--close  to  50  percent  (Bureau 
of  the  Census,  1982). 
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Traditionally,  the  federal  investment  in  transferring  scientific 
and  technical  information  (STI)  focuses  on  its  production  and  use.  Of 
the  $6.4  billion  the  federal  government  expends  annually  on  STI,  federal 
and  nonf ederal  information  organizations  receive  only  about  25  percent. 
The  bulk  goes  to  support  the  use  and  authorship  of  information  (King, 
1977;  Science  Indicators,  1980).  The  federal  government  supported 
authorship  accounted  for  38  percent  of  all  STI  authorship  in  1975.  Only 
4  percent  of  1975 's  federal  STI  costs  went  to  publishing,  and  only  1 
percent  to  abstracting  and  indexing.  King's  research  shows  that  each 
federally  funded  R&D  project  produces,  on  average:  1.98  articles  in 
U.S.  journals,  1.56  research  reports,  1.11  other  written  products 
(books,  monographs,  nonjournal  articles),  and  3.03  oral  presentations  in 
the  United  States  (King,  McDonald,  and  Roderer,  1981,  p.  259). 

Tools  for  Disseminating  Federally  Funded  Research 

Support  for  nonfederal  dissemination  of  federally  funded  research 
has  a  long  tradition.  In  1961,  the  Federal  Council  for  Science  and 
Technology  established  a  governmentwide  policy  of  including  page  charges 
to  journals  in  federal  contracts  and  grants  (Atkinson,  1978) .  This  page 
charge  policy  provides  critical  support  to  many  journals.2 

In  addition  to  disseminating  research  results  through  journal  and 
book  publishers,  the  federal  government  established  the  technical  report 
as  another  primary  outlet  for  research  results.  Although  government 
technical  reports  began  in  the  1800s,  their  recent  proliferation  is  tied 
to  the  appearance  of  mission  agencies  in  the  1950s  and  1960s.  In  terms 
of  providing  comprehensive  information,  technical  reports  have  the 
advantage  of  no  size  limits.  Scientists  and  engineers  often  use  them  to 
include  full  details  on  experimental  design  and  data  output, 
particularly  since  they  constitute  the  formal  records  of  projects. 


2Current  draft  regulations  by  Congress's  Joint  Committee  on 
Printing  (JCP)  put  pressure  on  this  system  as  evasions  of  the  depository 
library  laws  and  request  funding  agencies  to  prepare  an  annual  waiver 
request  to  the  JCP  listing  titles  of  the  articles  and  journals  and  total 
payments  (U.S.  Congress,  Joint  Committee  on  Printing,  1983). 
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Technical  report  production  has  placed  new  demands  on  the  federal 
information  system.  Large  federal  information  clearinghouses  were 
established  in  the  1950s  to  archive,  index,  abstract,  and  disseminate 
technical  reports.  Among  the  most  prominent  of  these  is  the  National 
Technical  Information  Service  (NTIS),  begun  in  1945  as  the  Publications 
Board  to  collect  World  War  II  documents  captured  from  the  enemy  and 
disseminate  them  to  industry.  By  1950,  Congress  had  converted  the  Board 
into  the  Office  of  Technical  Services  with  responsibility  for 
disseminating  federally  funded  research  results.  In  1971,  the  newly 
structured  NTIS  became  a  national  source  for  unclassified  federal 
scientific  and  technical  output  in  a  variety  of  forms,  including 
computer  software,  technology  licensing,  and  data  tapes. 

These  central  clearinghouses  and  depositories  have  been  hampered  in 
their  archival  roles  by  the  erratic  compliance  of  contributing  agencies. 
For  example,  in  1976,  the  National  Science  Foundation  (NSF)  issued  a 
Staff  Memorandum  requesting  its  directorates  to  forward  research  output 
to  NTIS.  Compliance  with  this  directive  has  been  incomplete,  with  the 
bulk  of  NSF/NTIS  reports  coming  from  only  two  directorates  (Sanchez  et 
al.,  1981).  Without  strict  compliance  among  government  agencies, 
material  can  be  lost  permanently- -particular ly  report  literature  for 
which  alternative  archives  such  as  libraries  do  not  usually  exist. 

Support  for  other  document  delivery  systems  has  followed  more 
traditional  mechanisms.  Currently  the  federal  government  funds  three 
national  libraries,  3000  libraries  in  the  federal  network,  several 
statistical  centers,  and  many  other  activities  (King,  1977).  The 
Library  of  Congress  has  supported  the  development  of  library  networks  by 
sharing  cataloging  information  and  an  active  inter library  loan  system. 

Mission-oriented  research  agencies  such  as  the  Atomic  Energy 
Commission  (AEC,  now  part  of  the  Department  of  Energy  or  DOE),  the 
National  Aeronautics  and  Space  Administration  (NASA) ,  and  the  National 
Institutes  of  Health  (NIH)  have  also  developed  services  to  meet  the 
multidisciplinary  information  needs  of  their  staff  and  contractors. 

They  established  abstracting,  indexing,  and  document  delivery  services 
for  journal  articles,  books,  dissertations,  nontechnical  government 
publications,  conference  proceedings,  and  international  sources.  Often 
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using  private  sector  contractors,  they  designed  advanced  computerized 
information  systems,  thereby  giving  federal  information  systems  a 
leadership  role  in  advancing  electronic  information  retrieval. 

Indexing  and  Abstracting 

Scientists  conducting  basic  research  traditionally  relied  on  major 
journals  and  abstracting  and  indexing  tools  produced  by  professional 
societies.  With  the  information  explosion,  many  of  these  societies 
could  not  keep  up  with  the  literature's  growth.  From  1958  to  1976, 

NSF's  Office  of  Science  Information  Services  (OSIS)  underwrote 
technological  experiments  and  automation  programs  by  the  American 
Chemical  Society  (ACS),  the  American  Institute  of  Physics,  the  American 
Psychological  Association,  the  American  Geological  Institute,  and  other 
organizations.  By  1974,  NSF  funding  for  these  programs  declined,  but 
most  of  the  programs  and  services  continued  to  operate  effectively.3 

These  automation  efforts  usually  led  to  the  creation  of  machine- 
readable  databases.  With  the  appearance  of  commercial  online  search 
services,  such  as  Dialog,  BRS,  and  SDC ,  these  databases  became  online 
files  paying  royalties  to  the  professional  societies  producing  them 
(Atkinson,  1978) . 

Reassessment  of  the  Government's  Role 

Reexaminations  of  the  government's  role  as  an  information  provider 
have  been  widespread  in  the  1980s,  but  conflicts  between  policies 
emphasizing  information  availability  and  others  emphasizing  restriction 
of  output  for  trade  protection  and  efficiency  also  have  a  long  history. 
In  1969,  the  Department  of  Defense  established  a  schedule  of  user 
charges  for  its  Information  Analysis  Centers.  AEC  and  NASA  soon  began 
to  charge  for  microfiche  and  other  services.  NTIS  has  long  supported 
itself  under  a  full-cost-recovery  program  with  a  special  revenue  fund 
from  its  sales.  Charging  for  federal  information  services  has  been 
given  more  support  by  the  Office  of  Management  and  Budget  pressures  on 
many  government  agencies  to  recover  the  costs  of  their  services  from 
users . 

3  In  1976,  the  OSIS  responsibilities  were  transferred  to  the  Office 
of  Science  and  Technology  Policy  in  the  White  House. 
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Many  government  information  operations  have  been  left  in  a 
dangerous  quandary  by  conflict  between  policies  promoting  information 
availability  and  directives  to  economize.  The  lack  of  coordinated 
policy  mechanisms  to  represent  the  special  interests  of  scientific  and 
technical  information  could  pose  a  real  threat  to  the  continuity  of  its 
flow . 

Summary 

Since  World  War  II,  the  federal  government  has  sponsored  the 
publication  of  federally  funded  research,  supported  secondary  services 
to  document  and  retrieve  the  published  scientific  and  technical 
literature,  and  funded  information  science  and  the  development  of  new 
information  technologies.  Federal  contributions,  in  collaboration  with 
the  private  sector,  have  substantially  accomplished  the  objective  of 
coping  with  the  information  explosion.  A  full  range  of  primary  sources-- 
journals ,  books,  technical  reports  - -that  document  research  findings  are 
being  produced.  Secondary  sources -- indexes ,  abstracts,  newsletters, 
trade  journals - -now  cover  most  major  areas  of  science  and  have  proven 
themselves  capable  of  expanding  quickly  to  cover  new  fields.  Advanced 
information  services--bibliographic  databases,  online  searching,  full- 
text  document  delivery,  information  brokers--are  emerging  from  the 
private  sector  with  some  government  support,  providing  worldwide  access 
to  the  scientific  and  technical  literature.  However,  recent  budgetary 
constraints  have  created  apprehension  about  continued  federal  support 
for  established  services.  The  lack  of  coordinated  policies  to  represent 
the  interests  of  the  scientific  and  technical  community  in  these  debates 
increases  the  concern  of  those  committed  to  the  integrity  of  this  body 
of  information. 

QUALITY  ASSURANCE 

Although  the  federal  government  has  made  available  enormous  amounts 
of  information,  the  quality  of  what  reaches  users  often  leaves  much  to 
be  desired.  For  more  than  twenty  years,  critics  have  noted  the  lack  of 
high  quality,  well-evaluated,  and  useful  information  from  federal 
research  efforts.  As  the  Weinberg  report  stated  in  1969:  "We  shall 
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cope  with  the  information  explosion,  in  the  long-run,  only  if  some 
scientists  and  engineers  are  prepared  to  commit  themselves  deeply  to  the 
job  of  sifting,  reviewing,  and  synthesizing  information:  i.e.,  to 
handling  information  with  sophistication  and  meaning,  not  merely 
mechanically.  Such  scientists  must  create  new  science,  not  just  shuffle 
documents.  Their  activities  of  reviewing,  writing  books,  criticizing, 
and  synthesizing  are  as  much  a  part  of  science  as  is  traditional 
research."  The  issue  has  not  changed.  Fifteen  years  later,  Branscomb 
(1983,  p.  11)  observed,  "Funding  of  research  to  produce  generally  useful 
knowledge  carries  with  it  a  responsibility,  not  only  to  ensure 
publication  of  results  but  also  their  evaluation  and  preparation  in  a 
form  suitable  for  application..." 

The  issue  of  quality  assurance  in  the  scientific  and  technical 
literature  has  generally  been  addressed  in  three  ways:  (1)  the  peer 
review  process,  (2)  published  compilations  or  reviews,  and  (3)  critical 
evaluation  programs  and  information  analysis  centers  for  some  physical 
and  chemical  data.  (The  discriminative  powers  of  advanced  information 
retrieval  services  and  databases  will  be  covered  in  the  section  on  User 
Responsiv. ) 

Peer  Review 

Peer  review  through  the  editorial  procedures  of  scientific  and 
technical  journals  offers  the  most  generally  applied  and  available 
quality  control  measure.  Federal  support  for  traditional  scientific 
dissemination  sources,  particularly  the  primary  journals,  has  had  the 
effect  of  applying  peer  review  to  federal  R&D  products.  Further,  some 
agencies  (e.g.,  NSF,  NIH)  encourage  publication  of  research  results  in 
reviewed  literature  rather  than  in  technical  reports.  Of  the  journal 
articles  submitted  in  a  year,  about  44  percent  are  rejected.  About  28 
percent  are  resubmitted  to  another  journal,  and  16  percent  are  dropped 
(King,  McDonald,  and  Roderer,  1981).  As  the  proliferation  of  journal 
titles  indicates,  the  chances  are  good  that  an  article  rejected  by  one 
journal  will  be  accepted  by  another. 

The  other  major  output  device  for  federal  research  is  the  technical 
report.  Report  literature  has  no  generally  established  review 
procedure.  For  this  reason,  among  others,  the  technical  reports  lack  a 
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reputation  for  reliability  among  most  basic  researchers.  On  the  other 
hand,  applied  researchers  and  engineers,  who  are  more  familiar  with 
them,  find  their  detail  and  currency  very  desirable. 

Passman  (1969)  points  out  that  technical  reports  may  in  fact 
receive  more  critical  review  than  is  generally  recognized.  Research 
proposals  have  undergone  peer  reviews,  and  the  researchers' 
qualifications  to  perform  the  research  have  been  scrutinized  before  the 
project  begins.  Many  federal  agencies,  particularly  those  supporting 
basic  research  like  the  National  Science  Foundation  and  the  National 
Institutes  of  Health,  have  long  established  peer  review  programs  to 
select  research  projects.  Throughout  the  course  of  the  project, 
researchers  may  consult  frequently  with  government  liaisons,  other 
groups  active  in  the  field,  and  even  advisory  groups.  When  the  project 
is  completed,  the  final  technical  report  may  be  reviewed  and  edited  by 
coworkers  within  the  research  organization,  often  including  professional 
editors  and  management  review  teams . 

Critical  Data  Evaluation 

Another  dimension  of  the  quality  assurance  picture  involves  the 
critical  evaluation  of  data  by  experienced  scientists.  The  objective  of 
such  an  evaluation  is  to  provide  authoritative  data  that  can  be  used 
with  confidence  by  a  well-defined  community. 

There  are  many  different  approaches  to  classifying  data.  For 
example,  the  International  Council  of  Scientific  Unions'  Committee  on 
Data  for  Science  and  Technology  (CODATA)  breaks  data  into  three  broad 
categories  with  numerous  subdivisions:  repeatable  measurements  on  well- 
defined  systems,  observational  data  on  environmental  systems,  and 
statistical  data  on  population  and  economic  characteristics  (Lide, 

1983).  Classification  could  also  follow  use  patterns:  data  used 
widely,  data  generated  and  used  only  by  specialists  in  the  same  field, 
and  data  used  by  scientists  in  a  limited  number  of  related  disciplines 
(Watson,  1983) . 

Evaluation  programs  vary  depending  on  the  type  of  data  collected 
and  the  users'  needs  among  other  things.  However,  most  programs  have 
four  things  in  common:  (1)  data  collection  (e.g.,  satellite  images, 
social  surveys,  literature  searches),  (2)  data  examination  (e.g., 
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experimental  techniques,  theoretical/calculational  processing,  measures 
and  standards  used),  (3)  critical  evaluation  by  experts  for  the  best 
values  (data  evaluation  centers,  critical  review  panels,  individual 
collectors),  and  (4)  dissemination  to  the  user  in  a  useful,  well- 
documented  form  (e.g.,  handbooks,  electronic  databases,  integrated 
design  analysis  programs) .  Most  evaluation  projects  require  expert 
staff  and  labor-intensive,  meticulous  work. 

Long-term  federal  support  for  critically  evaluated  or  reference 
data  started  with  the  creation  of  the  International  Critical  Tables 
project  in  1919,  administered  by  the  National  Research  Council  (NRC) . 

In  1957,  the  NRC  established  what  is  now  the  Numerical  Data  Advisory 
Board  (NDAB),  which  monitors  national  and  international  critical  data 
programs,  represents  the  United  States  on  CODATA,  and  provides  guidance 
to  the  National  Bureau  of  Standards  (NBS)  in  its  administration  of  the 
present  critical  tables  program.  The  other  major  federal  effort  in 
critically  evaluated  data  is  the  National  Standard  Reference  Data  System 
(NSRDS),  managed  by  NBS'  Office  of  Standard  Reference  Data  (OSRD) . 

NSRDS  coordinates  reference  activities  of  various  organizations  and 
establishes  standards  for  quality  and  methodology.  It  also  invests 
funds  to  establish  data  centers  and  compilation  projects,  but  only  when 
it  identifies  gaps  that  other  agencies  do  not  fill. 

Information  analysis  centers,  established  in  the  1960s  under  the 
impetus  of  the  Weinberg  report,  usually  provide  another  source  for 
evaluated  data.  Funded  by  government  or  industry,  examples  of  such 
centers  include  Battelle  Memorial  Institute's  centers,  the  Applied 
Physics  Laboratory  at  Johns  Hopkins  University,  the  Infrared  Research 
Information  Service  at  the  University  of  Michigan,  and  the  Electronic 
Properties  Information  Center  at  Purdue  University.  However,  only  some 
of  these  centers  are  still  involved  in  rigorous  critical  evaluation 
despite  the  fact  that  users  valued  the  centers'  evaluation  efforts  more 
highly  than  their  technology  transfer  activities  (Corridore,  1976). 

Other  government  agencies  have  programs  for  collecting  and 
evaluating  data:  The  Government-Industry  Data  Exchange  Program  (GIDEP) 
shares  technical  data  on  the  reliability  of  parts,  components,  and 
material  (Richards,  1977);  Brookhaven  National  Laboratory's  National 
Nuclear  Data  Center  coordinates  a  large  international  collection  of 
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atomic  chain  data;  the  National  Library  of  Medicine's  Toxicological  Data 
Bank  uses  a  network  of  scientists  to  review  the  toxicological  literature 
and  extract  and  evaluate  key  data;  and  the  Laboratory  Animal  Data  Bank 
records  detailed  information  on  the  strain  characteristics  of  control 
animals . 

Summary 

Despite  a  long  history  of  complaints  about  the  quality  of  data  from 
federal  projects  that  reach  users,  efforts  to  improve  quality  controls 
have  had  relatively  little  effect.  Technical  reports  rarely  receive 
review,  although  implicit  review  processes  may  be  in  place.  Data 
evaluation  programs,  which  dominate  the  literature,  receive  little 
funding  or  leadership  support.  Although  computer-based  technology 
promises  new  tools  for  providing  greater  amounts  of  evaluated 
information,  start-up  efforts  and  costs  have  prevented  their  more 
extensive  employment  for  this  purpose. 

USER-RESPONSIVENESS 

In  the  face  of  the  postwar  information  explosion,  federal 
policymakers  attempted  to  reach  scientists,  engineers,  and  industry  by 
establishing  technical-report  delivery  systems,  supporting  the  creation 
of  technologically  advanced  indexing  and  abstracting  systems,  and 
offering  low-cost  access.  However,  it  soon  became  apparent  that  formal 
information  channels  could  not  by  themselves  meet  the  needs  of  users . 

The  new  tools  were  not  readily  incorporated  into  ongoing  work.  "Another 
aspect  of  the  operation  of  the  communication  network  which  we  need  to 
understand  more  fully  is  the  sluggishness  with  which  individual  work 
habits  respond  to  new  opportunities  or  challenges"  (SATCOM,  1969, 
p.  103)  . 

As  a  first  step  toward  helping  users  to  search  the  masses  of 
primary  literature  for  specific  data,  a  number  of  data  f lagging  or  data 
tagging  projects  have  been  attempted.  Data  flagging  notes  the  presence 
of  primary  numeric  data,  while  data  tagging  offers  a  description  of  it. 
These  numerical  indexing  attempts  have  not  succeeded  in  attracting 
continuous  support  and  integration  into  information  services. 

Experiments  with  private  sector  information  services  such  as  Chemical 
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Abstracts,  with  journal  publishers  such  as  the  American  Institute  of 
Physics,  and  with  federal  information  services  such  as  NASA  have  not 
proved  fruitful.  The  cost  of  data  tagging- -equivalent  to  another 
subject  indexing  (Murdoch,  1980) --has  not  been  seen  as  justified  by  the 
market.  Proposals  to  change  the  structure  of  federal  research  reports 
to  include  data  tags  have  also  failed.  Several  professional  societies, 
for  example,  the  ACS  and  the  AIP,  have  established  data  repositories  for 
authors  of  articles  to  store  the  details  of  their  research  findings. 
Poorly  indexed  and  advertised,  these  depositories  received  minimal  use 
(Carter,  1980)  . 

In  attempts  to  get  information  into  users'  hands  more  quickly, 
agencies  began  to  support  user-based  information  networks.  In  1961,  the 
National  Institutes  of  Health  inaugurated  Information  Exchange  Groups  to 
provide  a  central  clearinghouse  for  exchanges  of  correspondence  and 
commentaries  with  colleagues  in  advance  of  journal  publication.  By 
1966,  the  system  was  sending  out  1.5  million  copies  of  preprints.  It 
was  so  successful  that  it  aroused  the  anger  of  established  publishers, 
support  was  withdrawn,  and  the  groups  dissolved  (Passman,  1969,  p.  71). 
The  advent  of  electronic  publishing  has  inspired  a  revival  of  this 
approach. 

Systems  such  as  the  above  still  built  on  formal  information  tools. 
Almost  as  soon  as  they  began,  the  major  government  information  services 
inaugurated  programs  to  reach  out  to  industry  and  later  to  the 
scientific  community  as  well.  Most  federal  agencies  had  chartered 
mandates  to  transmit  needed  information  to  their  constituent 
communities,  particularly  information  that  resulted  from  federally 
funded  research. 

With  the  development  of  automated  information  handling  techniques, 
most  of  the  major  clearinghouses  offered  a  varied  package  of  specialized 
services  by  the  early  1960s.  These  packages  included  continuing  current 
bibliographies  in  topical  areas,  searches  on  demand,  current-awareness 
programs,  technical  notes  alerting  researchers  to  major  new  findings, 
and  special  abstracts  for  new  research  areas.  For  example,  in  1965,  the 
Office  of  Education  established  the  Educational  Resources  Information 
Center  (ERIC),  which  employs  a  network  of  university-based 
clearinghouses  to  collect  and  disseminate  education  literature.  In 
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1966,  NASA  began  working  with  the  Lockheed  Corporation  to  design  an 
online  interactive  search  service  called  RECON.  By  1969,  the  system  was 
operating  with  seven  online  stations  and  a  library  of  a  half  million 
citations.  The  AEC  and  its  successor,  the  Department  of  Energy,  devised 
their  own  versions  of  the  same  system.  The  National  Library  of  Medicine 
(NLM)  contracted  with  Systems  Development  Corporation  (SDC)  to  design 
online  search  software  to  provide  access  to  their  MEDLARS  (Medical 
Literature  Analysis  and  Retrieval  System) .  Many  of  these  systems  became 
the  bases  for  commercial  ventures  using  software  originally  developed 
under  government  contract.  Most  clearinghouses  continue  these  services 
today.  However,  recent  pressure  on  the  budgets  of  federal  information 
services  and  programs  to  reduce  duplication  have  moved  many  agencies  to 
use  central  interagency  clearinghouses--particularly  the  National 
Technical  Information  Service--to  deliver  documents  to  the  public. 
Centralization  can  achieve  economies,  but  it  can  also  diminish  the 
communication  between  the  research  agency  and  its  technical  community. 

New  Technology 

With  the  arrival  in  the  1970s  of  government  and  commercial  online 
search  services,  new  discriminatory  capabilities  became  available.  More 
sophisticated  information  retrieval  was  offered,  at  least  in  principle, 
by  the  ability  to  design  search  strategies  more  tailored  to  individual 
user  needs  and  by  using  the  full  information  content  of  the  index  or 
abstract  record.  However,  most  databases  were  the  result  of  automating 
printed  services.  Few  were  redesigned  to  exploit  the  capabilities  of 
the  new  technology.  In  addition,  the  creators  of  the  online  systems 
faced  the  problem  of  integrating  information  from  different  databases 
that  used  different  standards  and  identification  schemes. 

In  addition  to  the  expense  and  complexity  of  online  searching  of 
databanks,  users  complain  that  it  is  difficult  to  assess  the  reliability 
of  the  data  without  sufficient  documentation  of  the  original  source 
(Rossmassler  and  Watson,  1980). 

Full-text  document  delivery  promises  new  avenues  for  locating  data. 
In  1983,  BRS  Inc.,  an  online  search  service,  added  all  eighteen  of  the 
American  Chemical  Society's  journals.  They  have  also  begun  to  add  the 
full  text  of  biomedical  and  psychological  textbooks,  e.g.,  Gray's 
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Anatomy  and  Mental  Measurements  Yearbook .  A  major  software  revision 
announced  by  Dialog,  the  largest  bibliographic  search  service,  will 
include  full-text  capabilities.  Recent  experiments  indicate  up  to  60 
percent  success  in  locating  articles  by  specific  data  needs  through  full- 
text  searching  techniques  (Rossmassler  and  Watson,  1980).  However, 
these  approaches  reach  only  a  handful  of  the  sources,  do  not  evaluate 
the  quality  of  the  data  found,  and  are  at  present  rather  costly. 

Although  online  search  services  were  originally  meant  to  serve  the 
information  user  directly,  intermediaries  continue  to  predominate  in 
online  searching.  However,  with  the  rise  of  microcomputers  and 
distributed  data  processing  in  this  decade,  opportunities  for  end-user 
searching  of  online  files  grow.  New  software,  such  as  Franklin 
Institute's  OL'SAM  and  ISl's  Searchmate,  may  encourage  direct  user 
access  to  bibliographic  and  numeric  databases. 

Directed  Information  Transfer 

In  the  1970s,  major  legislation  began  to  include  provisions  for 
directed  information  transfer.  The  National  Environmental  Policy  Act  of 
1969  took  a  significant  step  toward  improving  access  to  technical 
information  by  requiring  that  federal  agencies  prepare  Environmental 
Impact  Statements  and  publicize  the  documents  available  for  public 
scrutiny.  Agencies  were  not  to  wait  for  requests  but  were  to  contact 
organizations  with  environmental  interests  to  apprise  them  of  potential 
negative  effects.  The  Energy  Reorganization  Act  of  1974  mandated  the 
collection,  preparation,  and  dissemination  of  information  on  energy 
alternatives  "to  provide  the  full  exchange  of  ideas  and  criticism  that 
is  essential  to... public  understanding." 

Incorporating  Human  Information  Networks  and  Change  Agents 

In  the  1960s,  high-level  commission  reports  (Weinberg,  1969, 

SATCOM,  1969)  began  to  recommend  that  information  systems  conform  to  the 
way  scientists  and  engineers  really  communicated,  as  established  through 
research.  That  meant  heavier  reliance  on  interactive  and  informal 
processes  of  communication  rather  than  document  delivery.  As  the 
Weinberg  report  observed,  "...knowledgeable  scientific  interpreters  who 
can  collect  relevant  data,  review  a  field  and  distill  information  in  a 
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manner  that  goes  to  the  heart  of  a  technical  area  are  more  help  to  the 
overburdened  specialist  than  is  a  mere  pile  of  relevant  documents" 
(Weinberg,  1969). 

The  Information  Analysis  Centers  of  the  1960s  and  1970s,  although 
still  heavily  reliant  on  formal  publication  of  their  evaluated  data, 
also  attempted  to  use  technical  professionals  as  technology  transfer 
agents,  hoping  to  build  networks  of  communication.  The  high  cost  of 
these  programs  made  them  vulnerable  to  funding  cuts.  Ineffective 
marketing  hampered  their  service  to  their  target  audience.  User  studies 
showed  that  handbook  compilations  of  evaluated  data  were  prized  more 
than  technology  agent  functions  (Corridore,  1976). 

Other  mechanisms  using  change  agents  sprang  up  in  the  1970s. 
Congress  established  the  Office  of  Technology  Assessment  to  review  and 
analyze  technical  questions.  The  National  Institute  of  Education  s 
National  Diffusion  Network  reviews  educational  research  projects  and 
then  promotes  in-service  training  to  educators  throughout  the  country 
(Neill).  The  Research  Applied  to  National  Needs  (RANN)  program  began  at 
NSF  to  target  problems  of  national  concern  and  coordinate  their 
technical  solution  with  the  private  sector.  In  1974,  NSF  supported  and 
coordinated  the  establishment  of  the  Federal  Laboratory  Consortium,  an 
informal  organization  of  approximately  200  of  the  largest  federal 
research  laboratories.  A  key  consortium  practice  was  the  support  by 
each  member  laboratory  of  a  technology  transfer  representative  to  keep 
the  laboratory  in  touch  with  other  research  institutions  and  other 
federal,  private,  and  public  agencies.  The  goal  of  the  group  was  to 
provide  a  framework  for  applying  laboratory  capabilities  to  national 
problems  (Linsteadt,  1980).  The  Stevenson-Wydler  Act  of  1980  mandated 
such  technology  transfer  agents  for  every  laboratory,  but  funding  has 
not  been  forthcoming  (Gottlieb,  1982). 

University-industry  collaborative  research  programs  began  in  the 
1970s  as  an  effort  to  transfer  information  by  getting  knowledge 
procedures  and  users  together.  These  efforts  have  experienced  a  new 
surge  in  the  1980s,  especially  in  high-technology  areas,  where  research 
advances  rapidly  and  foreign  competition  threatens.  Declining 
university  budgets  and  enrollments  as  well  as  industrial  productivity 
concerns  have  encouraged  these  developments.  NSF  has  a  long  tradition 
of  promoting  this  type  of  venture  with  varying  success. 
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Summary 

Tailoring  information  to  target  audiences  requires  detailed 
knowledge  of  their  institutional  environment  and  of  the  way  in  which 
they  typically  use  information.  New  advances  in  information  technology 
may  in  principle  improve  the  ability  to  communicate  government  funded 
research,  but  in  fact  institutional  barriers  and  human  behavior 
considerations  still  predominate.  Changing  behavior  must  be  a  long¬ 
term  goal,  and  federal  information  policies  have  often  lacked  the 
direction,  support,  and  leadership  necessary  to  maintain  continuity. 
Improving  the  utility  of  collected  information  through  better 
technology,  better  access  structures,  enhancement  of  informal  systems  or 
a  combination  of  factors  is  possible.  However,  the  success  of  future 
efforts  is  likely  to  depend  on  the  extent  to  which  users  themselves  are 
involved  in  policy  and  system  development  instead  of  remaining  the 
passive  targets. 
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APPENDIX  C.  MILESTONES 

INFORMATION  POLICY  MILESTONES 

RELATED  EVENTS 

1945 

Publications  Board  established  by 
executive  order  to  collect  and  declas¬ 
sify  WWII  technical  data  for  dissemi¬ 
nation  to  industry. 

National  Advisory  Coommittee  on 
Aeronautics  initiated  scientific 
reports  to  supplement  scientific 
journals  for  dissemination  of  research 
results . 

1946 

1946 

Vannevar  Bush  called  for  establish¬ 
ment  of  National  Science  Foundation 
in  Science- -The  Endless  Frontier . 

First  all-electronic  digital  computer 
invented. 

Information  Center  on  Chemical- 
Biological  Coordination  established. 

National  Research  Council  Committee 
on  Line  Spectra  of  the  Elements 
established  sponsoring  atomic 
energy  levels  data  project  at 

National  Bureau  of  Standards 

1947 

1947 

Atomic  Energy  Commission  established 
Industrial  Information  branch. 

Microwave  radio  relay  system  commer¬ 
cially  operated  between  Boston  and 

N.Y. 

1950 

1950 

Law  enacted  establishing  Office  of 
Technical  Services  in  Department  of 
Commerce  to  serve  as  clearinghouse  for 
federally  sponsored  research  results 

First  automatically  sequenced  high¬ 
speed  electronic  digital  computer 
introduced  in  U.S. 

Passage  of  National  Science 

Foundation  Act. 

Berkner  Report  to  State  Department 
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on  science,  foreign  relations, 
and  national  flow  of  sci/tech 
information  released. 

Medical  Science  Information  Exchange 
(MSIE)  established  in  National 

Research  Council  to  inventory  current 
R&D  in  medicine. 

1951 

1951 

Armed  Services  Technical  Information 
Agency  established. 

First  UNIVAC  system  delivered  to 
Census  Bureau. 

1952 

NSF  established  Office  of  Science 
Information. 

National  Research  Council's 

Subcommittee  on  Physical  Chemistry 
issued  recommendations. 

1953 

NSF  shifted  Medical  Science  Informa¬ 
tion  Exchange  to  Smithsonian,  expanded 
scope  to  include  biology  and  psychol¬ 
ogy  and  renamed  it  Bio-Sciences 
Information  Exchange. 

1954 

1954 

Atomic  Energy  Act  authorized  AEC  to 
make  information  available  to  U.S. 
industry  and  foreign  countries  under 
cooperative  agreements. 

UNIVAC  commercially  available. 

1955 

Gas  Processors  Association  began 
Cooperative  Industry  Research 
program  to  improve  data  on  basic 
thermodynamic  properties  of  natural 
gas  liquids. 

1956 

1956 

U.S.  Armed  Forces  Medical  Library 
became  National  Library  of 

Medicine . 

First  voice  trans -Atlantic  telephone 
cable  with  underwater  amplifiers. 

Creation  of  President's  Science 
Advisory  Committee  and  position 
of  White  House  Science  Advisor 
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1957 

Information  Center  on  Chemical- 
Biological  Coordination  discontinued. 

National  Research  Council  esta¬ 
blished  Office  of  Critical  Tables 
(later  renamed  the  Numerical  Data 
Advisory  Board  (NDAB)). 

1958 

National  Defense  Education  Act 
created  Science  Information  Council 
and  Office  of  Science  Information 
Service  in  NSF. 

1959 

Federal  Council  for  Science  and 
Technology  established. 

1960 

Bio-Sciences  Information  Exchange 
became  Smithsonian  Science  Information 
Exchange  covering  ongoing  federally 
funded  R&D  projects  plus  some  private 
research  funding. 

DOD  established  Mechanical  Properties 
Data  Center. 

1961 

Federal  Council  for  Science  and 
Technology  approved  policy  of 
including  page  charges  by  scientific 
and  technical  journals  in  contract 
and  grant  costs . 

Committee  on  Scientific  Information 
established  in  FCST. 

1962 

Office  of  Science  and  Technology 
created  in  Executive  Office  of 
President . 

Library  of  Congress  organized  National 
Referral  Center  for  Science  and 


1958 

Second-generation  computers  using 
solid  state  circuitry,  stored  pro¬ 
grams,  and  user-oriented  programming 
language . 


Nasa  launched  its  first  active 
communications  satellite,  SCORE. 

1960 

NASA  launched  its  first  passive 
communications  satellite,  ECHO  I. 

American  Institute  of  Chemical 
Engineers  and  American  Petroleum 
Institute  established  the  Design 
Institute  for  Physical  Property  Data 
at  Pennsylvania  State  University. 


1962 

First  live  television  programs 
relayed  between  America  and  Europe 
via  Telstar  I . 

Depository  Library  Act  enacted. 
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Technology  to  review,  monitor,  and 
offer  coordinated  access  to  federal 
R&D  activities. 

Scientific  and  Technical  Communica¬ 
tions  in  the  Government  (Crawford 
Report)  recommended  central  agency  to 
direct  federal  information  policy  and 
establishment  of  special  units  in  R&D 
mission  agencies  to  deal  with  sci/tech 
information. 

NASA  contracted  with  American 
Institute  of  Aeronautics  and 
Astronautics  to  acquire  all  nonfederal 
published  literature  in  aeronautics 
and  astronautics  and  to  catalog, 
index,  and  abstract  it. 

1963 

Release  of  Science ,  Government  and 
Information:  The  Responsibilities  of 
the  Technical  Community  and  the 
Government  in  the  Transfer  of  Informa¬ 
tion  (Weinberg  Report)  recommended 
establishment  of  information  analysis 
centers  to  evlauate  research  data  and 
facilitate  transfer  of  information. 

National  Bureau  of  Standards 
established  National  Standard 
Reference  Data  System  to  improve 
critically  evaluated  data. 

1964 

Federal  Council  on  Science  and 
Technology's  Committee  on  Scientific 
Information  renamed  Committee  on 
Scientific  and  Technical  Information 
(COSATI) . 

Office  of  Technical  Service  became 
Clearinghouse  for  Federal  Scientific 
and  Technical  Information  (CFSTI) . 

Responsibility  for  coordination  of 
federal  sci/tech  information  activi¬ 
ties  transferred  from  NSF  to  Office  of 
Science  and  Technology  in  White  House. 


1964 

Federal  Property  and  Administrative 
Services  Act  of  1949  amended  (Brooks 
Act);  directed  General  Services 
Administration  to  coordinate  and 
provide  for  federal  computer  procure' 
ment . 

Chemical  Abstracts  Services  began  a 
systematic  registration  system  for 
all  chemical  compounds  (CAS  Registry 
Numbers) . 
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1965 

OECD  established  Scientific  and 
Technical  Information  Policy  Group. 

Medical  Library  Assistance  Act 
authorized  creation  of  a  regional 
medical  library  network  by  the 
National  Library  of  Medicine 

Office  of  Education  established 
Educational  Resources  Information 
Centers  (ERIC) . 

Federal  Library  Committee  organized. 

NSF,  DOD,  NLM  initiated  special 
funding  to  develop  advanced 
information  services  by 
professional  societies'  abstracting 
and  indexing  services. 

Multiyear  contract  let  to 
American  Chemical  Society  for 
automating  Chemical  Abstracts 
by  National  Science  Foundation. 

Lister  Hill  Center  for  Biomedical 
Communications  inaugurated  at  NLM  to 
support  pioneer  work  in  application 
of  computer  and  communications 
technology  to  biomedical -information 
transfer . 


1967 

NLM  established  Toxicology  Information 
Program . 

AEC  and  NASA  inaugurated  joint  program 
publishing  Tech  Briefs  to  reach 


1965 

Third-generation  computers  introduced, 
using  integrated  circuits, 
miniaturization,  system  logic 
improvements,  and  higher  speeds. 

Communications  Satellite  Corporation 
launched  Early  Bird,  first  in  its 
Intelsat  international  satellite 
communications  system. 


1966 

First  pilot  Federal  Information  Center 
opened  in  Atlanta. 

International  Council  of  Scientific 
Unions  estabished  Committee  on  Data 
for  Science  and  Technology  (CODATA) . 

Rare  Earth  Information  Center 
established  by  rare  earth  industry. 


smaller  firms  with  research  results. 


1968 

Congress  passed  Standard  Reference 
Data  Act. 


1969 

National  Academy  of  Science/National 
Academy  of  Engineering's  Committee 
on  Scientific  and  Technical 
Communication  (SATCOM)  produced 
final  report. 

DOD  required  Information  Analysis 
Centers  to  charge  for  products  and 
services . 

EPA-NIH  Chemical  Information  System 
(CIS)  established  as  collaborative 
international  effort  to  collect 
organic  chemical  databases  for 
online  networking  and  manipulation. 

1970 

CFSTI  became  National  Technical 
Information  Service  (NTIS)  with  clear¬ 
inghouse  functions  expanded  to  include 
business  and  statistical  information. 

U.S.  National  Commission  on  Libraries 
and  Information  Services  created. 


1968 

Carterfone  Decision--FCC  opened 
terminal  devices  for  connection  to 
the  telephone  system. 

1969 

MCI  Decision--FCC  authorized  provision 
of  private  line  microwave  services  by 
carriers  other  than  established 
common  carriers. 


National  Oceanographic  and  Atmospheric 


Agency  (NOAA)  established  Environmen¬ 
tal  Data  Service. 

1971 

NLM  and  Oak  Ridge  National  Laboratory 
initiate  Toxicology  Information 
Response  Center. 

1972 


1971 

Permanent  charter  for  International 
Telecommunications  Satellite  Corp. 
(INTELSAT)  signed  by  54  nations. 

1972 


ARPANET,  experimental  national  network  First  domestic  communication 
of  computers,  became  operational.  satellite  went  into  orbit. 
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1973 

NBS,  American  Chemical  Society,  and 
American  Institute  of  Physics  began 
publication  of  the  Journal  of 
Physical  and  Chemical  Reference 
Data . 

1974 

Federal  Laboratory  Consortium 
established  to  link  in-house  federal 
research  results  to  civilian  sector. 

Office  of  Education  established 
National  Diffusion  Network  to 
evaluate  and  promote  dissemination 
of  exemplary  programs  to  school 
districts . 


1976 

NSF  discontinued  Office  of  Science 
Information  Services  and  created 
Division  of  Science  Information  in 
Directorate  for  Scientific,  Techno¬ 
logical,  and  International  Affairs, 
refocusing  activities  to  support 
information  science  research. 

1978 

NSF  Division  of  Science  Information 
restricts  support  for  direct 
dissemination  of  sci/tech  information, 
placing  responsibility  on  research 
directorates . 

National  Telecommunications  and 
Information  Administration  formed  in 
Department  of  Commerce. 


1973 

Fluid  Properties  Research  Inc. 
established  at  Oklahoma  State 
University  by  international  group 
of  companies  to  provide  organic 
chemical  properties  data. 

1974 

Privacy  Act  of  1974  enacted. 


1975 

Freedom  of  Information  Act  amendments 
enacted . 

Low-cost,  limited-capacity  micro¬ 
computers  introduced. 

1976 

Computer  Software  Copyright  Act  enact¬ 
ed  . 


1978 

Federal  Information  Centers  Act 
enacted  establishing  national  network 
of  FICs. 


1980 


1980 
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Center  for  Utilization  of  Federal  Paperwork  Reduction  Act  enacted 

Technology  established  in  NTIS  by  establishing  OMB  as  agency  to 

Stevenson-Wydler  Technology  Innovation  oversee  federal  data  collection 

Act  to  provide  short  technology  procedures. 

assessment  reports  on  projects  with 

potential  use  by  the  private  sector  or 

local  governments . 

Integrated  Library  System  offered  for 
sale  to  library  community  by  NTIS 
after  two  years'  development  by  Lister 
Hill  Center. 

Laboratory  Animal  DataBank  available 
online  through  NLM  and  Battelle- 
Columbus  Laboratories. 

1980 

Administration  budget  cuts  eliminated 
several  statistical  collections-- 
Multiple  Jobholding  Survey  (BLS), 

Directory  of  Union  Membership  (BLS), 

Student  Residence  and  Migration 
Survey  (NCES),  Intercensal  Population 
Estimates  (Census). 

1982 

NLM  Toxicology  Information  Program 
initiates  Hazardous  Substances 
Information  Services  to  provide 
relevant  toxicological  information 
to  health  officials. 

1983  1983 

Joint  Committee  on  Printing  draft  OMB  revises  Circular  A-76  requiring 

regulations  redefined  government  agencies  to  contract  out  a  broad 

publication  to  include  electronic  range  of  administrative  activities, 

formats  as  well  as  print. 

Super index,  master  index  to  contents 
of  scientific  and  technical  handbooks, 
goes  online  commercially. 

Smithsonian  Science  Information 
Exchange  discontinued. 

1984 

AT&T  divested  itself  of  Bell  System 
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regional  phone  companies . 
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